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PREFACE 


This  publication  deals  with  the  quality  and 
quantity  of  scientific  research  publications  of  some 
3,000  scientists  in  the  Agricultural  Research  Serv- 
ice. It  is  based  on  responses  to  a  questionnaire 
sent  to  all  scientists  in  ARS  who  were  in  grades 
GS-7  and  above. 

The  author  received  extensive  help  and  guidance 
from  many  colleagues  in  ARS.  George  W.  Irv- 
ing, Jr.,  M.  W.  Parker,  H.  A.  Rodenhiser,  F.  R. 
Senti,  O.  W.  Herrmann  in  the  Office  of  Admin- 
istrator; J.  P.  McAuley,  A.  L.  Sykes,  M.  R.  Sev- 
erinson,  W.  F.  Leffler,  T.  W.  Little  in  the  Person- 
nel Division;  K.  A.  Tabler  in  Biometrical  Serv- 
ices; H.  TV.  Johnson,  D.  E.  DeTray  in  the  Animal 
Disease  and  Parasite  Research  Division;  E.  G. 
McKibben,  W.  M.  Carleton  in  the  Agricultural 
Engineering  Research  Division;  R.  E.  Hodgson, 
N.  D.  Bayley  in  the  Animal  Husbandry  Research 
Division;  H  R.  Thomas,  V.  R.  Boswell,  H.  O. 
Graumann  in  the  Crops  Research  Division;  E.  F. 
Knipling  in  the  Entomology  Research  Division; 
C.  H.  Wadleigh,  T.  W.  Edminster  in  the  Soil  and 
Water  Conservation  Research  Division;  W.  T. 
Pentzer,  H.  T.  Cook  in  the  Market  Quality  Re- 
search Division;  W.  C.  Crow,  R.  TV.  Hoecker  in 
the  Transportation  and  Facilities  Research  Divi- 
sion ;  Faith  Clark  in  the  Consumer  and  Food  Eco- 
nomics Research  Division;  Avis  TVoolrich  in  the 
Clothing  and  Housing  Reseai-ch  Division;  TV.  A. 
Gortner,  C.  Edith  Weir  in  the  Human  Nutrition 
Research  Division;  P.  A.  Wells  in  the  Eastern 
Utilization  Research  and  Development  Division; 
C.  IT.  Fisher  in  the  Southern  Utilization  Research 
and  Development  Division;  R.  J.  Dimler  in  the 
Northern  Utilization  Research  and  Development 
Division;  M.  .1.  Copley,  Fred  Stift  in  the  Western 
Utilization  Research  unci  Development. Division; 
and  D.  G.  Hall  in  the  Information  Division,  all 
assisted  in  the  development  of  the  questionnaire. 


Miss  Edith  LeRoy  in  the  Administrative  Serv- 
ices Division  was  principally  responsible  for  the 
design  of  the  questionnaire. 

Joel  Bitman,  Dean  F.  Davis,  P.  H.  Heinze,  J. 
M.  Lutz,  W.  L.  Smith,  Jr.,  C.  S.  Parsons,  B.  R. 
Burmester,  D.  F.  Stephens,  D.  L.  Brakensiek,  C. 
D.  Foy,  M.  H.  Frere,  L.  A.  Dean,  W.  A.  Gilbert, 
J.  A.  Alford,  R.  J.  Davey,  R.  E.  Hargrove,  G.  M. 
Sidwell,  M.  J.  Pallansch,  O.  W.  Parks,  and  Miss 
Virginia  H.  Holsinger,  all  scientists  of  ARS,  par- 
ticipated in  a  pretest  of  the  questionnaire  and 
offered  many  helpful  suggestions  for  its  im- 
provement. 

The  Personnel  Evaluation  Committees  in  the 
Agricultural  Engineering  Research  Division,  Ani- 
mal Husbandry  Research  Division,  Animal  Dis- 
sease  and  Parasite  Research  Division,  Crops 
Research  Division,  Entomology  Research  Division, 
Soil  and  Water  Conservation  Research  Division, 
Eastern  Utilization  Research  and  Development 
Division,  Northern  Utilization  Research  and  De- 
velopment Division,  Southern  Utilization  Re- 
search and  Development  Division,  and  Western 
Utilization  Research  and  Development  Division, 
all  made  peer  evaluations  of  the  publications  of 
a  random  sample  of  scientists.  All  Division 
Personnel  Evaluation  Committees  offered  helpful 
suggestions  on  the  evaluation  plan  for  rating 
publications. 

E.  Fred  Schultz,  Jr.,  of  the  Biometrical  Serv- 
ices Staff  offered  helpful  suggestions  in  presenting 
the  tabular  and  graphic  material. 

Finally,  special  acknowledgment  must  go  to  the 
3,207  scientists  in  ARS  who  responded  to  the  ques- 
tionnaire. It  took  much  conscientious  effort  on 
their  part  to  complete  this  rather  complex  ques- 
tionnaire. 
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The  Use  of  Quality  and  Quantity  of  Publication  as 
Criteria  for  Evaluating  Scientists 

Byron  T.  Shaw,  Assistant  to  Administrator, 
Agricultural  Research  Service, 
U.S.  Department  of  Agriculture 

SUMMARY 


The  complete  publication  records  to 
January  1,  1965,  of  some  3,000  scientists 
in  the  Agricultural  Research  Service 
(ARS)  were  considered.  The  scientists 
ranged  in  age  from  19  to  69  and  in  GS 
grades  from  7  to  18.  Forty-four  and 
three-tenths  percent  of  the  scientists  had 
doctor's  degrees;  28  percent  had  master's 
or  professional  medical  degrees ;  26.2  per- 
cent had  bachelor's  degrees;  and  1.5  per- 
cent had  no  degree. 

The  total  number  of  publications  per 
scientist  ranged  from  0  to  278.  Thirteen 
hundred  scientists  had  10  or  less  publica- 
tions ;  1,426  had  between  11  and  50  publi- 
cations; 255  had  between  51  and  100 
publications;  44  had  over  100  publica- 
tions; and  4  had  over  200  publications. 

When  the  scientists  were  separated  by 
age,  there  was  still  a  wide  variation  in  the 
number  of  publications  per  scientist. 
Even  at  age  67,  the  spread  was  from  4 
to  239  publications.  When  quality  of 
publications  is  considered  along  with  quan- 
tity of  publications  to  get  a  publication 
score,  there  is  still  a  widespread  difference 
between  scientists  of  the  same  age.  (Total 
publication  score  =  the  summation  of  the 
products  of  publication  credit  (from  0.1 
to  1.0— depending  upon  the  number  of 
authors)  and  rank  order  (quality  meas- 
ure from  1  to  100)  for  individual  papers.) 


Over  half  of  the  scientists  (1,594)  had 
total  publication  scores  of  less  than  300; 
1,266  of  these  had  scores  under  200  and 
806  had  scores  under  100 ;  yet  9  scientists 
had  scores  over  3,000. 

The  mean  number  of  publications  and 
mean  publication  score  were  calculated 
for  scientists  of  the  same  age  from  age  22 
to  age  69.  The  mean  number  of  publica- 
tions ranged  from  0  at  age  22,  5.2  at  age 
30,  14.4  at  age  40,  27.2  at  age  50,  39.8  at 
age  60,  to  65.2  at  age  69.  The  mean  pub- 
lication score  ranged  from  0  at  age  22, 
71.9  at  age  30,  250  at  age  40,  569  at  age 
50,  851  at  age  60,  to  1,108  at  age  69. 

Publication  score  per  year  is  a  better 
measure  than  total  publication  score  for 
use  in  comparing  scientists,  since  publica- 
tion score  per  year  eliminates  the  vari- 
ables of  age  and  length  of  experience. 
The  mean  publication  score  per  year 
ranged  from  1.8  for  scientists  in  grade  7, 
13.6  for  scientists  in  grade  11,  31.6  for 
scientists  in  grade  13,  to  45.7  for  scientists 
in  grade  15. 

Scientists  with  doctor's  degrees  had  a 
mean  number  of  publications  per  year  of 
1.68  and  a  mean  publication  score  per 
year  of  31.4.  Scientists  with  master's  or 
professional  medical  degrees  had  a  mean 
number  of  publications  per  year  of  1.24 
and  a  mean  publication  score  per  year  of 
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20.4.  Scientists  with  bachelor's  degrees 
had  a  mean  number  of  publications  per 
year  of  0.89  and  a  mean  publication  score 
per  year  of  13.8. 

Scientists  who  have  been  publishing  for 
21  or  more  years  had  a  mean  number  of 
publications  per  year  of  1.68  and  a  mean 
publication  score  per  year  of  31.2.  Sci- 
entists who  have  been  publishing  from  11 
to  20  years  had  a  mean  number  of  pub- 
lications per  year  of  1.51  and  a  mean 
publication  score  per  year  of  26.8.  Scien- 
tists who  have  been  publishing  from  0  to 
10  years  had  a  mean  number  of  publica- 
tions per  year  of  1.09  and  a  mean  publi- 
cation score  per  year  of  17.6.  Contrary 
to  generally  held  views,  there  was  a  steady 
increase  in  publication  output  with  age. 

Publications  per  year  and  publication 
score  per  year  were  highly  correlated  with 
promotions  for  those  scientists  who  have 
been  employed  continuously  by  ARS  since 
January  1,  1955. 

The  net  change  in  quality  of  ARS  sci- 
entists in  two  periods  was  calculated.  It 
was  found  that  there  was  a  small  net  de- 
crease in  quality  from  June  30,  1956,  to 
June  30,  1960,  and  a  small  net  increase 
in  quality  from  June  30, 1960,  to  June  30, 
1965.  These  differences  in  the  two  peri- 
ods are  believed  to  be  related  to  the 
change  to  the  "man  in  the  job"  concept 
of  classifying  scientists  which  was 
adopted  in  1959. 

ARS  does  a  good  job  of  selecting  re- 
search supervisors  from  scientists  con- 
siderably above  average  in  ability. 

ARS  pays  its  scientists  in  accordance 
with  their  productivity.  Those  scientists 
in  the  top  10  percentile  of  salary  for  their 


age  produced  six  times  as  many  publica- 
tions and  had  publication  scores  eight 
times  those  of  scientists  in  the  bottom  10 
percentile  of  salary  for  their  age.  The 
means  for  each  10  percentile  in  salary  for 
their  age  lie  in  a  straight  line  on  a  loga- 
rithmic scale. 

The  publications  of  89  scientists  se- 
lected at  random  were  evaluated  by  10 
peer  groups  representing  10  research  divi- 
sions. The  members  of  each  peer  group 
independently  evaluated  the  publications 
of  up  to  11  scientists  in  its  own  division. 
The  publications  of  each  scientist  were 
evaluated  independently  by  from  four  to 
seven  peers.  In  peer  groups  represent- 
ing divisions  having  a  narrow  range  of 
scientific  disciplines,  the  independent  peer 
evaluations  were  fairly  consistent  with 
each  other.  In  peer  groups  representing 
divisions  with  a  wide  range  of  disciplines, 
the  independent  peer  evaluations  varied 
widely.  However,  in  6  peer  groups, 
where  5  peers  in  each  group  independ- 
ently evaluated  the  publications  of  10 
scientists,  the  rank  orders  (based  on  pub- 
lication score  per  year)  assigned  the  10 
scientists  by  each  of  the  5  peers  were 
remarkably  consistent.  Three  of  the  six 
groups  had  a  wide  range  of  disciplines  and 
three  had  a  narrow  range. 

Finally,  based  on  the  foregoing  analysis 
and  with  much  advice  from  a  large  num- 
ber of  scientists,  a  plan  for  evaluating 
scientific  research  publications  is  pre- 
sented. The  plan  deals  with  who  should 
make  the  evaluations,  the  division  of 
credit  among  authors,  and  the  assessment 
of  quality  of  publications. 


USE    OF    QUALITY    AND    QUANTITY    OF    PUBLICATION    FOR    EVALUATING    SCIENTISTS 

Introduction 


The  Agricultural  Research  Service  (ARS) 
classifies  and  promotes  its  scientists  in  accordance 
with  the  Civil  Service  Commission  Research 
Grade-Evaluation  Guide  (9).1  An  article  by  Dr. 
P.  A.  Wells  in  Research  Management  (10)  de- 
scribes this  guide  and  discusses  the  background 
of  its  development.  The  classification  and  promo- 
tion program  for  research  scientists  is  carried  out 
through  the  mechanism  of  Division  Personnel 
Evaluation  Committees.  These  committees  are 
composed  of  senior  research  scientists  with  back- 
grounds similar  to  the  scientists  being  evaluated 
and  a  personnel  classification  officer.  Dr.  Wells 
expressed  the  views  of  ARS  when  he  stated :  "We 
find  that  the  system  is  simple.  It  is  readily  under- 
stood. It  ranks  positions  in  proper  relationship, 
and  it  is  equitable  across  disciplinary  lines.  Our 
senior  scientists,  who  have  devoted  time  and  effort 
to  make  it  work,  feel  that  the  system  is  very  ef- 
fective and  even  though  a  substantial  amount  of 
time  in  study  and  committee  deliberation  is  re- 
quired, they  feel  the  end  result  justifies  the 
price.  .  .  .  The  system  is  not  perfect,  but  it  is  the 
most  important  advance  in  classification  since  the 
original  Classification  Act  was  passed  in  1923." 

In  an  effort  to  make  a  good  system  even  better, 
a  study  was  undertaken  to  "Develop  more  reliable 
criteria,  methods,  and  techniques  for  evaluating 
qualifications,  achievements,  and  professional  sta- 
ture of  scientists  in  ARS."  The  first  phase  of 
the  study,  which  is  reported  here,  deals  with  the 
evaluation  of  the  quality  and  quantity  of  a  scien- 
tist's research  publications.  It  considers  the  rela- 
tion of  the  scientist's  publication  output  to  his 
training,  age,  length  of  experience,  position  title, 
grade,  previous  employment,  work  location,  and 
Merit  Index.2  It  examines  the  correlation  between 
publication  output  and  promotions.  It  compares 
peer  evaluations  of  research  publications  with  a 


1  Italic  numbers  in  parentheses  refer  to  Literature 
Cited,  p.  50. 

3  Dr.  Shockley(7),  1956  Nobel  Prize  winner  in  physics, 
defines  Merit  Index  for  an  individual  for  a  given  date  as 
the  fraction  of  the  employees  of  his  age  that  the  individ- 
ual exceeds  in  salary.  Thus,  the  person  having  the  top 
salary  for  his  age  will  have  an  index  of  1.0;  the  person 
having  the  median  salary  will  have  an  index  of  0.5 ;  and 
the  person  having  the  lowest  salary  will  have  an  index  of  0. 


scientist's  own  evaluation.  It  also  evaluates  the  net 
change  in  quality  of  ARS  scientists  in  two  time 
periods.  Finally,  the  results  of  the  study  are  used 
to  develop  a  plan  for  evaluating  the  quality  and 
quantity  of  a  scientist's  research  publications. 

The  data  used  in  the  study  came  from  3,207 
responses  to  a  questionnaire  sent  to  all  research 
scientists  in  ARS  in  grades  GS-7  and  above.  This 
represented  responses  from  more  than  95  percent 
of  all  research  scientists  in  these  grades.  The  ques- 
tionnaire and  instructions  for  its  use  are  presented 
in  the  appendix.  Briefly,  the  questionnaire  re- 
quired the  respondent  to  evaluate  the  quality  and 
quantity  of  his  own  publications  on  the  basis  of 
guides  presented  in  the  instructions.  The  first 
guide  concerned  the  proportion  of  credit  each  au- 
thor on  multiple-author  papers  should  receive  (see 
appendix,  sec.  B).  Total  credit  added  to  1.0  for  a 
single  paper.  The  second  guide  related  to  the  qual- 
ity of  publications  based  on  the  impact  the  publica- 
tions had  or  could  be  expected  to  have  on  science, 
agriculture,  or  public  welfare.  (See  appendix,  sec. 
A  and  the  list  of  publications  on  pp.  60  to  74.) 
The  guide  consisted  of  a  set  of  publications,  which 
the  research  division  directors  in  ARS  had  given 
rank  orders  from  100  to  1,  that  the  scientist  could 
compare  with  his  own  publications  in  establish- 
ing their  rank  order. 

Using  these  guides,  the  respondent,  for  each  of 
his  papers,  determined  an  author  credit  value 
(from  0.1  to  1.0)  and  a  rank  order  (from  100  to  1). 
He  then  multiplied  the  credit  by  the  rank  order  to 
get  a  publication  score  for  that  paper 
(0.5X30=15).  Next,  he  added  the  values  on  all 
papers  for  given  time  periods  to  get  (1)  total  pub- 
lications, (2)  total  publication  credit,  and  (3)  total 
publication  score.  He  then  divided  these  totals 
by  the  years  in  the  time  period  to  get  per-year 
values. 

It  was  expected  that  some  scientists  would  over- 
rate their  publications  and  some  would  underrate 
theirs,  but  it  was  believed  that  means  determined 
from  fairly  large  number's  of  scientists  would  have 
a  reasonable  relation  to  peer  ratings.  Several 
scientists  sent  in  their  work  sheets  showing  the  rank 
order  assigned  to  each  of  their  papers.  It  ap- 
peared that  they  had  done  a  reasonable  job  of 
ranking. 
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Characteristics  of  the  Population  Completing  the  Questionnaire 


Of  the  3,207  scientists  who  responded  to  the 
questionnaire,  15  were  in  grades  16  and  above. 
Because  of  the  low  numbers  in  grades  16,  17,  and 
18,  the  analyses  in  this  paper  are  confined  to  grades 
7  through  15.  The  distribution  of  scientists  by 
grade  and  by  degree  is  shown  in  table  1. 

Leisner's  (4-)  figures  for  all  biologists  and  agri- 
cultural scientists  responding  to  the  1962  canvass 
by  the  National  Register  of  Scientific  and  Tech- 
nical Personnel  show  38.8  percent  with  doctor's 
degrees,  34.0  percent  with  master's  and  profes- 


sional medical  degrees,  and  25.6  percent  with 
bachelor's  degrees.  The  National  Science  Founda- 
tion reports  that  35  percent  of  all  scientists  on  the 
register  in  1964  had  doctor's  degrees,  30  percent 
had  master's  and  professional  medical  degrees, 
and  32  percent  had  bachelor's  degrees  (5) . 

The  ages  of  ARS  scientists  ranged  from  19  to 
69.  The  age  distribution  by  grade  and  by  degree 
is  shown  in  table  2  and  in  figures  1  and  2.  The 
distribution  of  scientists  by  grade  and  by  degree  by 
age  groups  is  shown  in  table  3. 


Table  1. — Number  and  percentage  distribution  of  ARS  scientists  by  grade  and  by  degree 


Degree 

Grade 

Total 

Percentage 

7 

9 

11 

12 

13 

14 

15 

distribution 

No  degree 

15 

131 

32 

1 

12 

198 

135 

6 

12 
219 
264 
155 

7 
140 
226 
511 

3 
91 

137 
405 

0 

40 

69 

197 

0 

20 

31 

135 

49 

839 

894 

1,410 

1.  5 

B.S.,  B.A 

26.  3 

M.S.,  M.A.,  D.V.M 

Ph.  D.,  Sc.  D 

28.0 
44.2 

Total . 

Percentage 

179 
5.6 

351 
11.  0 

650 
20.4 

884 
27.7 

636 
19.9 

306 
9.6 

186 
.     5.8 

3,  192 
100.  0 

100.  0 

Table  2. — Age  distribution  of  ARS  scientists 

by  grad 

?  and  by  degree 

Age 

Grade 

Degree 

Total 

7 

9 

11 

12 

13 

14 

15 

Ph.  D. 

M.S. 

B.S. 

None 

Over  65.       -  _       

3 
1 

1 
0 

0 

1 

0 
0 
2 
0 
0 

0 
0 
0 

1 

4 

1 
2 

1 
4 

1 

2 
2 
2 
0 
2 
0 
1 
2 
2 
5 
2 

4 
4 
2 
6 
4 
5 
3 
2 
3 
9 

7 
4 
0 
3 
2 
7 
Ki 
5 
7 
6 
6 

11 
6 
11 
9 
13 
14 
18 
15 
12 
10 

17 

3 

14 

5 

9 

8 

15 

12 

13 

10 

11 

10 
10 
14 
19 
15 
16 
15 
21 
30 
25 

16 

7 

6 

8 

12 

10 

20 

10 

10 

12 

16 

11 
6 
16 
16 
17 
12 
23 
14 
22 
19 

12 

5 

5 

6 

8 

9 

11 

16 

10 

9 

12 

11 
10 

8 
14 
16 
19 

9 
12 
12 
11 

11 
3 
3 
6 
8 
9 
9 
5 
7 

10 
8 

10 
9 
9 
9 
5 

11 
8 

12 
5 
6 

29 
11 
12 
13 
21 
19 
35 
21 
20 
11 
27 

17 
13 
22 
23 
33 
39 
34 
41 
34 
35 

21 
8 
12 
9 
11 
12 
16 
14 
17 
23 
11 

16 
15 
19 
30 
18 
17 
16 
16 
33 
21 

16 

6 

7 

5 

8 

13 

15 

14 

13 

17 

15 

21 
16 
16 
19 
22 
20 
27 
19 
21 
23 

2 

0 
0 

1 
1 
0 
0 

1 
1 
1 

2 

3 

1 
3 
2 
1 
2 

T 

i 

0 
2 

68 

65.. 

25 

64.. 

31 

63  . 

28 

62 

41 

61 

44 

60 

66 

59 

50 

58 

51 

57.. 

52 

56 

55 

55 
57 

54.. 

45 

53 

52 

60 
74 

51.. 

74 

50.. 

78 

49.. 

78 

48.. 

77 

47 

46 

88 
81 
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Table  2. — Age  distribution  of  ARS  scientists  by  grade  and  by  degree — Continued 


Age 

Grade 

Degree 

Total 

7 

9 

11 

12 

13 

14 

15 

Ph.  D. 

M.S. 

B.S. 

None 

45- 

3 
5 
0 
3 
1 
2 
4 
2 
4 
0 

3 

7 

9 

6 

10 

10 

10 

15 

9 

15 

17 

22 

9 

13 

7 

9 

4 

6 

11 

11 

8 

10 

20 

25 

21 

8 

23 

15 

23 

27 

20 

11 

5 

3 

24 
21 
11 
21 
35 
30 
25 
29 
36 
22 

28 

32 

32 

29 

29 

18 

23 

17 

9 

3 

0 

0 

36 
36 
42 
27 
40 
32 
32 
26 
52 
46 

41 

33 

33 

31 

22 

25 

13 

14 

6 

4 

1 

0 

35 
38 
40 
26 
34 
22 
26 
24 
25 
25 

18 
18 
9 
5 
6 
1 
1 
0 
0 
0 
0 
0 

11 
16 

10 

7 

11 

6 

4 

7 
5 

1 

2 

1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

2 
9 
3 
5 
0 
2 
1 
0 
0 

1 

0 
0 
0 
0 
0 
0 

0 

0 
0 
0 
0 
0 

53 
60 
60 
49 
63 
45 
48 
51 
79 
68 

63 

58 

48 

41 

33 

32 

23 

19 

4 

3 

0 

0 

31 
48 
33 
24 
34 
28 
31 
22 
31 
21 

31 
27 
26 
19 
25 
20 
19 
27 
22 
11 
8 
1 

34 
29 
19 
24 
27 
24 
22 
25 
19 
15 

18 
29 
30 
19 
31 
16 
26 
27 
18 
20 
15 
19 

2 
1 
1 
1 
1 
3 
2 
1 
1 
0 

0 
2 
0 

0 

1 
1 

2 

0 
0 
0 
0 
5 

120 

44.. 

138 

43.. 

113 

42  . 

98 

41.. 

125 

40.. 

100 

39 

38 

103 
99 

37 

130 

36.. 

105 

35 

112 

:;i 

116 

33.. 

104 

32  . 

79 

31.. 

90 

30.. 

69 

29.. 

70 

28 

73 

27.. 

44 

26.  _ 

33 

25 

23 

Under  25 

25 

Table  3. — Number  oj  scientists  by  grade  and  degree,  by  age  group 


Age  and  degree 

Grade 

Total 

7 

9 

11 

12 

13 

14 

15 

Under  28  years 
No  degree .  .  

5 
51 

7 
0 

0 
17 
22 

0 

0 
4 
6 
2 

0 
0 
7 
4 

0 
0 
0 
0 

0 
0 
0 
0 

0 
0 
0 
0 

5 

B.S.,  B.A 

72 

M.S.,  M.A.,  D.V.M 

42 

Ph.  D.,  Sc.  D 

6 

Total... 

63 

39 

12 

11 

0 

0 

0 

125 

28-32  years 
No  degree .  

0 
41 
10 

0 

4 
57 
34 

1 

0 
18 
54 

44 

0 

3 

11 

91 

0 
0 

1 

12 

0' 
0 
0 
0 

0 
0 
0 
0 

4 

B.S.,  B.A 

119 

M.S,.  M.A.,  D.V.M 

Ph.  D.,  Sc.  D 

110 
148 

Total 

51 

96 

116 

105 

13 

0 

0 

381 

33-37  years 

No  degree 

3 

15 

5 

0 

0 
48 
36 

0 

0 
37 
58 
55 

0 

10 

27 

168 

0 

1 

10 

84 

0 
0 
0 
9 

0 
0 
0 

1 

3 

B.S.,  B.A 

111 

M.S.,  M.A.,  D.V.M... 

136 

Ph.  D.,  Sc.  D 

317 

Total 

23 

84 

150 

205 

95 

9 

1 

567 
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Table.  3. — Number  of  scientists  by  grade  and  degree,  by  age  group — Continued 


Age  and  degree 

Grade 

Total 

7 

9 

11 

12 

13 

14 

15 

38-42  years 

No  degree   .                                 _   _ 

3 
6 
3 
0 

1 
20 
18 

2 

3 
57 
54 
26 

1 
29 
37 

90 

0 

9 

21 

102 

0 

1 

6 

28 

0 
0 
0 
8 

8 

B.S.,  B.A 

122 

M.S.,  M.A.,  D.V.M 

Ph.  D.,  Sc.  D 

139 
256 

Total 

12 

41 

140 

157 

132 

35 

8 

525 

43-47  years 

No  degree ______                        _ 

1 
9 
2 

1 

1 
23 
16 

1 

4 
30 
31 
13 

0 
36 
57 
76 

0 
21 
34 

99 

0 

5 

20 

35 

0 

2 

6 

17 

6 

B.S.,  B.A 

126 

M.S.,  M.A.,  D.V.M 

166 

Ph.  D.,  Sc.  D 

242 

Total .... 

13 

41 

78 

169 

154 

60 

25 

540 

48-52  years 

No  degree 

3 
4 
2 
0 

2 

15 

3 

0 

0 
33 

28 
8 

1 
22 
25 
38 

1 
21 
17 
43 

0 

9 

15 

46 

0 

3 

7 

35 

7 

B.S.,  B.A 

107 

M.S.,  M.A.,  D.V.M 

97 

Ph.  D.,  Sc.  D 

170 

Total 

9 

20 

69 

86 

82 

70 

45 

381 

53-57  years 

No  degree 

0 
0 
0 
0 

2 

12 

3 

0 

2 
18 

17 
3 

4 
18 
24 

9 

2 
17 
19 
23 

0 
12 
12 
26 

0 

8 

9 

29 

10 

B.S.,  B.A 

85 

M.S.,  M.A.,  D.V.M 

84 

Ph.  D.,  Sc.  D 

90 

Total 

0 

17 

40 

55 

61 

50 

46 

269 

58-62  years 

No  degree. 

0 
2 
1 
0 

2 
2 
3 
0 

1 

20 

9 

1 

0 
14 
22 
21 

0 
12 

19 
31 

0 

9 

10 

35 

0 

4 

6 

28 

3 

B.S.,  B.A 

63 

M.S.,  M.A.,  D.V.M 

70 

Ph.  D.,  Sc.  D 

116 

Total 

3 

7 

31 

57 

62 

54 

38 

252 

Over  62  years 

No  degree   -            -- - 

0 
3 
2 
0 

0 
4 
0 
2 

2 
2 

7 
3 

1 

8 

16 

14 

0 

10 
16 
11 

0 

4 

6 

18 

0 

3 

3 

17 

3 

B.S.,  B.A 

34 

M.S.,  M.A.,  D.V.M 

50 

Ph.  D.,  Sc.  D. 

65 

Total 

5 

6 

14 

39 

37 

28 

23 

152 
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Figure  1. — Age  distribution  of  ARS  scientists — all  and  by  degree. 
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Figure  2. — Age  distribution  of  ARS  scientists  by  grade. 


Individual  Variation  in  the  Number  and  Quality  of  Research  Publications 


Since  in  this  and  following  sections  of  the  report 
some  responses  could  not  be  used,  the  number  of 
respondents  will  differ  from  those  given  in  the 
previous  section.  The  principal  reason  for  not 
using  responses  was  that  publication  records  asked 
for  were  for  the  period  before  January  1,  1965, 
and  a  number  of  scientists  were  hired  between 
that  time  and  August  16,  1965,  when  the  ques- 
tionnaire was  sent  out. 

It  is  well  known  that  in  any  group  of  scientists 
the  number  of  publications  per  scientist  will  vary 
widely.    ARS  scientists  follow  the  general  trend. 


The  total  number  of  publications  per  scientist 
ranged  from  0  to  278.  Thirteen  hundred  scientists 
had  10  or  less  publications;  1,426  had  between 
11  and  50  publications;  255  had  between  51  and 
100  publications;  and  44  had  over  100  publica- 
tions. A  frequency  distribution  showing  the  num- 
ber of  scientists  by  grade  having  a  given  number 
of  publications  is  given  in  table  4.  The  percentage 
distribution  of  scientists  having  a  given  number 
of  publications  is  given  by  grade  in  table  5  and 
the  cumulative  percentage  is  shown  in  table  6  and 
figure  3. 
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Table  4. — Number  of  scientists  hating  a  given  number  of  publications,  total  and  by  grade 


Publications 

Grade 

Total 

Cumulative 

7 

9 

11 

12 

13 

14 

15 

total 

n 

71 

22 

16 

8 

13 

5 

3 

4 

56. 
39 
48 
37 
22 
22 
18 
18 
14 
7 

4 

27 

8 

6 

1 
1 

32 
35 
41 
40 
49 
45 
42 
47 
29 
29 

22 

89 

49 

29 

18 

7 

6 

3 

3 

1 

18 
16 
21 
25 
37 
46 
34 
38 
41 
36 

40 
150 
99 
73 
54 
25 
22 
17 
8 

7 
4 
4 
5 

1 

2 

2 

9 

3 

6 

10 

12 

11 

16 

23 
90 
92 
85 
58 
44 
37 
22 
22 

22 
10 
8 
9 
3 
1 
4 
3 
3 

178 
115 
128 
120 
124 
124 
107 
121 
97 
94 

92 

379 

278 

229 

173 

129 

98 

69 

71 

59 
38 
35 
37 
25 
13 
15 
11 
15 
7 

6 
6 
4 
3 
4 
2 
3 
2 
1 
2 

4 
3 
3 

1 

1__    . 

1 

293 

2 

421 

3 

1 

541 

4 

665 

5 

789 

6 

896 

7__ 

1 

1 
2 

1 
14 
22 
27 
32 
40 
22 
15 
24 

20 
15 
12 
12 
12 
9 
5 

3 

1 

3 

1 
1 

1 

2 

6 

6 

9 

10 

12 

11 

11 

14 

9 

9 

10 

11 

10 

2 

6 

6 

9 

6 

1 
3 
3 

2 
2 

1 
3 

1 

3 
2 
1 

1 

1,017 

8              .   

1,  114 

9  . 

3 

1,208 

10                     .  -. 

1,300 

11  to  15  .           ...      

3 

2 

1,679 

16  to  20 ...      ... 

1,957 

21  to  25                       

2,  186 

26  to  30     ..                

2,359 

31  to  35 

2,488 

36  to  40 

2,586 

41  to  45 

1 

2,656 

46  to  50 

2,  726 

2,785 

56  to  60 

2,823 

61  to  65 

1 

2,858 

66  to  70 

2,895 

71  to  75 

2,  920 

76  to  80. 

1 

2,933 

81  to  85 

2,948 

86  to  90 

1 

1 

2,959 

91  to  95 

2,974 

96  to  100 

2,981 

101  to  105  ..     .    .   

2 
3 

2,987 

106  to  110  ... 

2,993 

Ill  to  115 

2,  997 

116  to  120 

3,000 

121  to  125.  .. 

1 

3,004 

126  to  130 

3,  006 

131  to  135 

3,  009 

136  to  140 

3,  011 

141  to  145 

3,012 

146  to  150 

1 

3,014 

151  to  175.. 

3,  018 

176  to  200 

3,  021 

201  to  250.. 

1 

1 

3,  024 

251  to  300 

3,  025 

Total.. 

150 

329 

618 

824 

615 

303 

186 

3,025 
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Table  5. — Percent  of  scientists  having  a  given  number  of  publications,  total  and  by  grade 


Publications 

Grade 

Total 

7 

9 

11 

12 

13 

14 

15 

0        .   .              .         -     _ 

47.33 
14.67 
10.  67 
5.33 
8.  67 
3.33 
2.  00 
2.  67 

17.02 

11.  85 

14.  59 

11.25 

6.  69 

6.  69 

5.47 

5.47 

4.  26 

2.13 

1.22 

8.21 

2.  43 

1.82 

.30 

.30 

5.  18 
5.66 
6.63 
6.47 
7.  93 
7.28 

6.  80 
7.61 
4.  69 
4.69 

3.  56 
14.  40 

7.  93 

4.  69 
2.  91 
1.  13 

.97 
.49 
.49 

.  16 

2.  18 

1.  94 

2.  55 

3.  03 

4.  49 

5.  58 
4.  13 
4.61 
4.98 
4.  37 

4.85 
18.20 
12.  01 

8.86 

6.  55 
3.  03 
2.  67 
2.  06 

.97 

.85 
.49 
.49 
.  61 

0.  16 
.33 
.33 

1.46 
.49 
.98 

1.63 

1.  95 
1.79 
2.60 

3.  74 

14.  63 

14.  96 

13.82 

9.43 

7.  15 

6.  02 

3.58 

3.  58 

3.  58 

1.  63 

1.  30 

1.46 

.49 

.  16 

.65 

.4& 

.49 

5.  88 

1 

0.33 

3.  80 

2        ._.   ..   .   

4.  23 

3          -   

0.  54 

3.97 

4        .   .       . 

4.  10 

5        ------ 

4.  10 

6 

3.  54 

7 

.33 
.33 
.66 

.33 
4.  62 
7.26 
8.91 
10.  56 
13.20 
7.26 
4.  95 
7.  92 

6.60 
4.  95 
3.96 
3.  96 
3.  96 
2.  97 
1.  65 

.54 
.54 
.  54 

1.  OS 
3.23 
3.23 
4.84 
5.38 
6.45 
5.  91 
5.  91 
7.53 

4.84 
4.84 
5.  38 
5.  91 
5.  38 
1.  08 
3.23 
3.23 
4.84 
3.23 

.  54 
1.61 
1.  61 
1.08 
1.  08 

.54 
1.61 

.  54 

4.  00 

8      --   

3.21 

9        . 

2.  00 

3.  11 

10        ------ 

3.  04 

11  to  15 

2.  00 
1.33 

12.  53 

16  to  20     -   . 

9.  19 

21  to  25      _. 

7.  57 

26  to  30     - 

5.72 

31  to  35   __ 

4.26 

36  to  40_    -. 

3.24 

41  to  45 -. 

.30 

2.28 

46  to  50 

2.  35 

51  to  55 

1.  95 

56  to  60---    -- 

1.  26 

61  to  65 

.  16 

1.  16 

66  to  70  .      .. 

1.22 

71  to  75 

.  83 

76  to  80 

.  12 

.43 

81  to  85..- -   -.- 

.  50 

86  to  90 

.  16 

.  12 

.  36 

91  to  95--- --   -- 

.  99 
.33 

.99 

.50 

96  to  100 

.23 

101  to  105 

.33 
.49 

.  20 

106  to  110 

.20 

111  to  115   - 

.33 
.33 
.33 
.33 

.  13 

116  to  120   

.  10 

121  to  125 

.  16 

.  13 

126  to  130 

.  07 

131  to  135 

.  10 

136  to  140 

.33 
.33 
.  33 

.33 
.33 

.  07 

141  to  145 . 

.  03 

146  to  150 

.  12 

.  07 

151  to  175... 

1.61 

1.  08 
.54 
.  54 

.  13 

176  to  200 -. 

.  10 

201  to  250 

.  12 

.  16 

.  10 

251  to  300 

.  03 

Total 

100.  00 

100.  00 

100.  00 

100.  00 

100.  00 

100.  00 

100.  00 

100.  00 
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Table  6. — Cumulative  percentage  of  scientists  having  a  given  number  or  fewer  publications,  total  and  by 

grade 


Publications 

Grade 

Total 

7 

9 

11 

12 

13 

14 

15 

0          

47. 
62. 
72. 
78. 
86. 
90. 
92. 
94. 

33 

00 
67 
00 
67 
00 
00 
67 

17.02 
28.  87 
43.46 
54.71 
61.40 
68.09 
73.  56 
79.03 
83.29 
85.42 

86.  64 
94.  85 
97.28 
99.  10 
99.40 
99.70 

5.  18 
10.  84 
17.  47 
23.94 
31.  87 
39.  15 
45.  95 
53.  56 
58.25 
62.94 

66.50 

80.90 
88.83 
93.  53 
96.43 

97.  56 

98.  53 
99.02 

99.  51 

99.  67 

2.  18 

4.  12 

6.  67 

9.  70 

14.  19 

19.  77 

23.  90 

28.51 

33.49 

37.  86 

42.  59 
60.  79 
72.  93 
81.79 
88.  34 
91.  37 
94.  04 

96.  10 
97.07 

97.  92 

98.  41 

98.  90 

99.  51 

0.  16 

.49 

.82 

2.  28 

2.  79 

3.75 

5.  38 

7.  33 

9.  12 

11.  72 

15.46 
30.09 
45.  05 
58.87 
68.  30 
75.45 
81.47 
85.  05 
88.  63 

92.  21 

93.  84 

95.  14 

96.  60 

97.  09 

97.  25 
97.90 

98.  39 
98.88 

5.  88 

1__ 

0.33 

9.  68 

2 

13.  91 

3.. 

0.54 

17.  ss 

4. 

21.  98 

5.. 

26.  08 

6.. 

29.  62 

7 

.  66 

.99 

1.  65 

1.  98 
6.  60 
13.86 
22.  77 
33.33 
46.53 
53.79 
58.74 
66.  66 

73.26 
78.21 
82.  17 
86.  13 
90.  09 

93.  06 

94.  71 

1.08 

1.  62 

2.  16 

3.24 
6.  47 
9.  70 
14.  54 
19.92 
26.37 
32.28 
38.  19 
45.72 

50.  56 
55.40 
60.78 
66.69 
72.07 
73.  15 
76.38 
79.  61 
84.45 
87.68 

88.  22 
89.83 
91.44 

92.  52 

93.  60 

94.  14 

95.  75 
96.29 

33.  62 

8..      .... 

36.  83 

9.- 

96. 

67 

39.  94 

10 

42.  98 

11  to  15-- 

98. 
100. 

67 
00 

55.  51 

16  to  20.. 

64.  70 

21  to  25. . 

72.  27 

26  to  SC- 

77. 99 

SI  to  35-- 

82.  25 

36  to  40-. 

85.  49 

41  to  45.. 

100.  00 

87.  77 

46  to  50.. 

90.  12 

51  to  55 

92.07 

56  to  60 

93.  33 

61  to  65 . . 

99.83 

94.  49 

66  to  70.. .   . 

95.  71 

71  to  75 

96.  54 

76  to  80      ._.__.     .   ... 

99.  63 

96.  97 

81  to  85.    -   -.   .     .   .   - 

97.  47 

86  to  90. 

100.  00 

99.75 

97.  83 

91  to  95-- 

95.70 

96.  03 

97.02 

~~97.~35~ 

97.  68 

98.  01 
98.34 

98.  33 

96  to  100   .     - 

98.  56 

101  to  105.    

99.21 
99.  70 

98.  76 

106  to  110 .-. 

98.  96 

111  to  115     

99.  09 

116  to  120.. 

99.  19 

121  to  125 . 

99.86 

99.  32 

126  to  130 

99.  39 

131  to  135. 

99.  49 

136  to  140 

98.67 
99.  00 
99.33 

99.66 
100.  00 

99.  56 

141  to  145 

99.  59 

146  to  150 

99.87 

99.  66 

L51  to  175 

97.90 

98.  98 

99.52 

100.  00 

99.  79 

176  to  200 

99.  89 

201  to  250 

100.  00 

100.  00 

99.  99 

2.51  to  300 

100.  00 

12 
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Table  6  and  figure  3  bring  out  some  interesting 
relationships.  It  would  appear  that  the  following 
relationships  between  the  number  of  publications 
and  the  percentage  of  scientists  in  different  grades 
approximate  the  facts : 

50  percent  of  the  scientists  in  one  grade  have 
as  many  publications  as  or  more  publications 
than  25  percent  of  the  scientists  in  the  next 
higher  grade. 

25  percent  of  the  scientists  in  one  grade  have 
as  many  publications  as  or  more  publications 
than  50  percent  of  the  scientists  in  the  next 
higher  grade,  and  25  percent  of  the  scientists 
in  the  second  higher  grade. 

10  percent  of  the  scientists  in  one  grade  have 
as  many  publications  as  or  more  publications 
than  75  percent  of  the  scientists  in  the  next 
higher  grade,  50  percent  of  the  scientists  in 
the  second  higher  grade,  and  25  percent  of  the 
scientists  in  the  third  higher  grade. 

3  percent  of  the  scientists  in  one  grade  have 
as  many  publications  as  or  more  publications 
than  90  percent  of  the  scientists  in  the  next 
higher  grade,  75  percent  of  the  scientists  in 
the  second  higher  grade,  50  percent  of  the 
scientists  in  the  third  higher  grade,  and  25 
percent  of  the  scientists  in  the  fourth  higher 
grade. 

The  above  relationships  are  brought  out  more 
clearly  in  table  7. 


When  scientists  are  separated  by  age,  as  in  table 
8,  there  is  still  a  wide  variation  in  the  number  of 
publications  per  scientist.  Even  at  age  67  the 
spread  is  from  4  to  239  publications.  It  is  ap- 
parent that  the  more  prolific  scientists  produce  a 
substantial  proportion  of  the  total  publications. 
This  is  brought  out  even  better  in  table  9  where 
the  cumulative  percentage  of  authors  in  decreasing 
publication  rank  order  is  compared  with  cumula- 
tive percentage  of  publications.  Twenty  percent 
of  the  authors  produce  half  of  the  publications. 

When  quality  of  publications  is  considered 
along  with  quantity  of  publications  to  get  a  pub- 
lication score,  there  is  still  a  widespread  difference 
between  scientists  of  the  same  age.  This  is  shown 
in  table  10.  (Total  publication  score  =  the  sum- 
mation of  the  products  of  publication  credit  (0.1 
to  1.0)  and  rank  order  (1-100)  for  individual  pa- 
pers.) Over  half  of  the  scientists  (1,594)  had 
total  publication  scores  of  less  than  300 ;  1,266  of 
these  had  scores  under  200  and  806  had  scores  un- 
der 100 ;  yet  9  scientists  had  scores  over  3,000. 

All  scientists  who  had  publications  on  which 
they  were  sole  or  senior  author  were  asked  to  state 
the  highest  rank  order  they  assigned  to  any  one 
publication.  The  results  are  shown  in  tables  11 
and  12.  The  range  in  highest  rank  order  was 
from  1  to  100  with  a  mean  of  53. 
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NUMBER  OF  PUBLICATIONS 

Figure  3. — Cumulative  percentage  of  scientists,  grade's  GS-7  to  GS-15,  having  a  given  number  or  fewer  publications. 


Table  7. — Number  of publications  not  exceeded  by  the  given  percentage  of  scientists, 

by  grade  of  scientist 


Percentage  of 
scientists 


97 
90 
75 
50 
25 


Grade 


11^ 
5 

2 
1 
0 


20 

-11, 

6 

3 

1 


11 

34 
21 

7 
3 


12 

48 
-33, 

21 
-12, 

7 


13 

73 

51 
"35. 

21 
•13 


14 

115 
75 
57 

^38, 
26 


15 

155 

110 

80 

53 

^34 
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Table  8. — Number  of  scientists,  by  age,  having  a 


Number  of 

Age,  June  30,  1965 

publications  ' 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

0  to  4 - 

2 

4 

15 

1 

18 

1 

30 
1 
1 

44 
3 

2 

47 

7 
1 
2 

37 

19 

5 

47 

21 

7 

6 

.... 

37 

14 

7 

.... 

1 

41 
29 

7 
5 
3 

2 

1 

46 

39 

6 

7 
6 

42 
36 
19 
6 
5 
1 
2 
2 

31 
27 
15 
4 
4 
3 
2 
1 
1 

35 

34 

20 

11 

8 

5 

1 

3 

25 

21 

19 

10 

6 

3 

5 

2 

1 

18 
27 
23 
25 

7 
5 
4 

7 

33 

14 

15 

6 

8 

3 

14 
30 
22 
20 

7 
4 
5 
4 

12 

22 

26 

18 

6 

8 

6 

5  to  9          -      _ 

10  to  14.    ._ 

15  to  19 

1 

20  to  24.    

1 

25  to  29.. 

30  to  34   .    

1 

35  to  39 

1 

40  to  44 

1 

1 
2 

6 

1 

1 
1 
1 
1 

45  to  49.    -.    

50  to  54. . 

1 

1 

~~2 

1 

55  to  59-. 

60  to  64.    .    

1 

1 



65  to  69..  . 

1 



70  to  74..    

1 

75  to  79 

1 

80  to  84 

85  to  89.-    

90  to  94   ._-    --. 

1 

95  to  99   --      ... 

100  to  104 . 

105  to  109 . 

110  to  114 

115  to  119   ...    _ 

120  to  124. _.    . 

125  to  129 

130  to  134 .. 

135  to  139 

140  to  144 

145  to  149.. 

150  to  154 

155  to  159.  . 

160  to  164 

165  to  169 

170  to  174   . 

175  to  179      

185  to  189.. 

195  to  199 

200  to  204 

205  to  209. 

235  to  239 

275  to  279 

1  For  total  career  of  scientist  regardless  of  employment. 
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Age,  June  30,  1965 — Continued 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

65 

66 

67 

68 

69 

15 

14 

16 

18 

15 

8 

4 

2 

2 

4 

1 
1 

13 

18 

22 

22 

16 

6 

3 

10 

4 

4 

1 
1 

9 

31 

27 

9 

14 

18 

8 

3 

7 

2 

5 

10 

25 
19 
12 

5 
6 
2 
1 
4 

3 

8 
5 

15 
8 

11 
3 
7 
6 
5 
3 

4 
3 

1 

7 

12 
7 
9 

13 
7 
6 
4 
5 
4 

1 

3 

.... 

2 
1 

4 
7 
9 
6 
9 
7 
3 
4 
6 
1 

2 

1 
1 

"2 
2 
2 

6 
6 

12 
7 

11 
4 
3 
8 
5 

4 

5 

13 

7 

8 

10 

10 

4 

3 

4 

9 

8 
9 
11 
8 
6 
4 
5 
1 
1 

5 
4 
2 

~~2~ 
1 

3 
4 
6 
4 
7 
5 
4 
3 
4 
3 

3 
2 
4 
5 

1 
4 
8 
4 
4 
4 
8 
3 
2 

2 
2 
2 
1 
2 
1 
1 

3 
5 
6 
2 
5 
1 
9 
2 
2 
3 

"I" 

1 
1 
2 
2 
1 
2 

5 
3 
3 
3 
6 
7 
2 
2 
5 
3 

4 
1 

1 
3 

"2" 

2 
2 
5 
2 
5 
6 
5 
2 
3 
1 

2 
2 
3 

"I" 
1 
3 
1 

1 
2 
7 
6 
6 
6 
4 
5 
1 
3 

2 
1 
3 
2 
1 

1 
2 
4 
3 
6 
3 
7 
2 
2 
3 

2 
1 
1 

"4" 

3 

7 
2 
4 
6 
8 
2 
7 

~6~ 

4 
2 

"2" 

2 
2 
1 
2 
3 
2 
3 
1 

2 
2 

1 
3 
1 
2 
1 

2 

5 
2 
7 
5 
5 
3 
4 
1 
4 

2 
1 
3 
4 
2 
1 
3 
1 
1 

1 
1 
5 
3 
5 
4 
4 
3 
1 

3 
3 

"I" 

1 

2 
3 
3 

4 
2 
2 
3 

2 

2 

2 
3 

1 

2 

4 
2 
1 
4 

1 

2 
2 

"l" 

3 

1 
4 

2 
2 

1 
2 

1 
1 

1 

1 

1 
1 

2 

"2" 

"2 

2 
2 

1 
1 

5 

1 

1 
1 
1 
1 

2 

T 

1 

1 

1 

~~3~ 
5 

2 
2 

1 
2 

1 
1 

1 

1 

1 

3 

2 

2 

1 
1 
1 

... 

1 

2 

... 

... 

1 

2 

1 

1 

1 
3 
1 
1 

3 

1 

.... 

1 

1 
2 

1 

1 
1 

2 

1 

1 

... 

2 

1 
1 

1 

3 
1 

1 

2 

1 

1 

1 

1 

— 

1 

1 

1 
1 

2 

1 

1 

1 

1 
1 

... 

1 

... 

1 

1 

... 

1 

1 

1 

1 

1 

1 

1 
1 

1 
1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 



1 

1 

1 

... 

1 

1 

1 

1 

234-109  O— GG 
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Table  9. — Comparison  of  cumulative  percentage  of  authors  in  decreasing  publication  rank  order  with 

cumulative  percentage  of  publications 


Cumulative  percentage  of 

Cumulative  percentage 
of  publications 

Cumulative  percentage  of 
authors  in  decreasing 
publication  rank  order 

Cumulative  percentage 
of  publications 

authors  in  decreasing 
publication  rank  order 

All 
respondents 

Respondents 

of  the  same 

age1 

All 
respondents 

Respondents 

of  the  same 

age1 

5  percent     .  _ 

22 
37 
47 
56 
70 
80 

20 
31 
40 
48 
61 
72 

50  percent 

87 
92 
96 

98.  5 
99.8 
100 

80 

10  percent                 . 

60  percent 

87 

15  percent 

70  percent 

93 

20  percent     __                    . 

80  percent 

97 

30  percent  __ 

90  percent 

99 

40  percent,  _ 

100  percent 

100 

Calculated  for  ages  43,  48,  53,  and  58,  all  of  which  showed  the  same  relationship. 

Table  10. — Number  of  scientists,  by  age,  having  a 


Total 
publication 

Age,  June  30,  1965 

score  ' 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

0  to  99.    

2 

4 

14 

13 

27 
1 

44 
1 
1 

45 
4 

42 

14 

1 

1 

56 

11 

4 

2 

38 

7 
4 

49 

15 

6 

3 

53 

24 
10 

~~2 

51 
23 
11 

7 
5 
3 

38 

23 

6 

4 
2 
1 

1 

44 

24 

13 

13 

6 

2 

1 

1 

24 

26 

15 

10 

5 

3 

3 

1 

1 

29 

21 

17 

13 

9 

5 

1 

1 

2 

1 

21 
22 
13 
9 
6 
3 
5 
3 

26 

17 

18 

20 

8 

4 

1 

1 

18 

23 

20 

12 

10 

3 

4 

2 

1 

100  to  199 

200  to  299 

1 

300  to  399 

400  to  499 

500  to  599 

600  to  699 

1 

700  to  799 

1 

800  to  899.  _ 

1 

1 
1 

900  to  999 .  _ 

1 

1 

1 

1,000  to  1,099 

1,100  to  1,199 

1 



1,200  to  1,299 

1,300  to  1,399 

1,400  to  1,499. 

1,500  to  1,599- 

1,600  to  1,699-. 

1,700  to  1,799 

1,800  to  1,899 

1,900  to  1,999 

2,000  to  2,099 

2,100  to  2,199 

2,200  to  2,299 

2,300  to  2,399 

2,400  to  2,499 

2,500  to  2,599 

2,600  to  2,699 

2,700  to  2,799 

2,800  to  2,899 

2,900  to  2,999 

3,000  to  3,099 

3,100  to  3,199 

3,200  to  3,299 

3,300  to  3,399 

3,400  to  3,499 

3,500  to  3,599 

3,600  to  3,699 

3,700  to  3,799 

1  For  total  career  of  scientist  regardless  of  employment. 
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given  total  publication  score  to  Jan.  1 ,  1965 


Age,  Junt 

!  30 

1965- 

Continued 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

65 

66 

67 

68 

69 

21 

18 
I 'J 
11 
8 
5 
7 
4 
1 
1 

3 

2 

17 

22 

24 

12 

11 

5 

3 

5 

2 

3 

3 

22 
21 
22 

14 
20 

9 
7 

7 

1 

17 
21 

12 

8 

11 

1 
7 
4 
1 

1 

3 

5 
14 
13 

9 

10 
5 
7 
4 
2 
2 

1 

11 
8 

15 
6 
6 
7 
7 
3 
5 
2 

2 
1 
1 
1 

9 
10 

5 
5 
2 
6 
5 
5 

~~4~ 

1 
.... 

1 

1 

10 
10 

9 
6 
6 
5 
4 
1 
2 
3 

6 
1 
3 
1 

7 
7 
8 
7 
6 
5 
11 
3 
7 
4 

1 

1 

~~2~ 

10 

10 

10 

12 

2 

5 

3 

3 

3 

2 

3 
2 
3 
1 
3 
1 

5 
8 
8 
6 
3 
6 
3 
5 

~3~ 

4 

3 

8 
7 
5 
3 
6 

"I" 

2 

9 

~3~ 
3 
3 
5 
1 
2 
2 
3 

4 

4 
7 
4 
5 
4 
5 
3 
2 
3 
2 

1 
3 

3 
4 
3 
3 
3 
4 

"5" 
4 
1 

1 
3 

1 
5 
5 
4 
5 
3 
7 
1 
2 
2 

1 
1 
2 
4 
3 

1 
4 
7 
4 
4 
3 
2 
5 
2 

5 
7 
5 
6 
2 
3 
4 
6 
5 
1 

4 
2 
1 
1 
1 
1 

"4" 
4 

7 
6 
2 
2 
3 
1 
4 

2 
2 
2 

3 

2 
1 
1 

1 

1 
1 

1 
1 
1 
5 
5 
6 
5 
3 
1 
2 

4 
1 

1 

2 
3 
4 

2 

"§" 
3 

2 
2 
1 

"I" 

1 
1 

1 

3 
2 
3 

4 

"4" 
2 

3 
3 

2 

1 
2 
1 

1 

2 
3 

4 
1 

"I" 

1 

"I" 

"I" 

2 

2 
3 
1 
3 
2 
2 

1 

1 

1 

1 

2 
1 
2 
1 

5 

1 
1 
2 

--- 

1 

~~i 

1 
3 

1 

"2" 
1 

1 
3 

1 

1 

2 

1 

1 

2 

1 

1 
1 

2 
1 

T 
1 

2 

1 
4 
3 

1 

1 
3 
1 

"I" 

2 
"I" 

1 
1 

1 
2 
1 
1 
1 
1 
1 

1 
3 

2 

2 

1 

1 

1 
2 

1 



1 
1 
1 

"I" 

~2 

1 
1 
2 
1 

2 

1 

1 

.... 

1 

1 

1 

1 
1 

1 

1 
2 

1 

1 
1 

2 
1 

1 

1 
2 



1 

1 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 
1 
1 

... 

1 

2 

1 

1 

... 

1 

1 

--- 

... 

1 
1 
1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

1 

1 

1 

1 

1 

1 
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Table  11. — Number  oj  scientists  having  a  given  highest  rank  order  on  sole  or  senior  author  papers,  total 

and  by  grade 


Rank  order 

Grade 

Total 

Cumulative 

7 

9 

11 

12 

13 

14 

15 

total 

1  to  10.   .     .- 

2 
5 

1 
1 
5 
3 

10 

15 

20 

37 

43 

38 

9 

9 

4 

13 

45 

68 

88 

123 

99 

45 

20 

9 

4 

2 

21 

63 

119 

201 

198 

77 

64 

14 

2 

6 

12 

29 

57 

131 

167 

80 

83 

20 

9 

33 
103 
197 
339 
568 
626 
305 
283 
73 
21 

33 

11  to  20 

2 
12 
23 
40 
85 
57 
64 
11 

1 

3 
4 
14 
25 
36 
37 
43 
15 
5 

136 

21  to  30  -. 

333 

31  to  40     .-_--. 

672 

41  to  50 

1,  240 

51  to  60  -.                     

1,  866 

61  to  70     

2,  171 

71  to  80                 -      

2,  454 

81  to  90  -         -_        .-   - 

2,  527 

91  to  100 

2,  548 

Total 

17 

185 

514 

761 

594 

295 

182 

2,548 

Table  12. — Cumulative  percent  oj  scientists  by  grade  having  a  given  highest  rank  order  or  less  on  sole  or 

senior  author  papers,  by  grade 


Rank  order 

Grade 

Total 

7 

9 

11 

12 

13 

14 

15 

1  to  10 

11.  8 

41.  2 
47.  1 
52.  9 
82.3 
100.  0 

5.4 
13.5 
24.  3 
44.  3 
67.6 
88.  1 
93.  0 
97.8 
100.  0 

2.  5 
11.3 
24.  5 
41.6 
65.6 
84.8 
93.6 
97.  5 
99.2 
100.  0 

0.  3 
3.0 
11.3 
26.9 
53.  4 
79.4 
89.  5 
97.9 
99.  7 
100.  0 

1.  0 

3.0 

7.9 

17.5 

39.6 

67.  7 

81.  1 

95.  1 

98.5 

100.  0 

1.3 

1 1  to  20 

0.  7 

4.8 

12.6 

26.  1 

54.9 

74.  2 

95.9 

-  99.7 

100.0 

1.  7 
3.9 
11.  5 
25.3 
45.  1 
65.4 
89.  0 
97.3 
100.  0 

5.3 

21  to  30     -- 

13.  1 

31  to  40-.-   

26.4 

41  to  50   - 

48.  7 

51  to  60      .   . 

73.2 

61  to  70   - 

85.  2 

71  to  80- 

96.3 

81  to  90-.     _-_ 

99.  2 

91  to  100.-   -..   

100.  0 

Publication  score  per  year  is  a  better  measure 
than  total  publication  score  to  use  in  comparing 
scientists,  since  publication  score  per  year  elimi- 
nates the  length-of-experience  variable.  Tables 
13,  14,  and  15  and  figure  4  give  number  and  per- 
centage of  scientists  by  grade  having  a  given  pub- 
lication score  per  year.  Again,  there  is  a  wide 
variation  among  scientists,  but  most  of  the  spread 
is  caused  by  about  10  percent  of  the  scientists. 
Ninety  percent  of  the  scientists  have  scores  per 
year  of  60  or  less,  whereas  the  highest  score  per 
year  is  250. 

The  relationships  among  grades  in  number  of 
publications  shown  in  table  7  hold  even  better  with 
publication  score  per  year;  these  relationships  are 
brought  out  in  table  16. 


It  would  appear  that  the  following  relationships 
between  publication  score  per  year  to  January  1, 
1965,  and  the  percentage  of  scientists  in  different 
grades  approximate  the  facts : 

50  percent  of  the  scientists  in  one  grade  have 
a  publication  score  per  year  as  high  as  or 
higher  than  25  percent  of  the  scientists  in  the 
next  higher  grade. 

25  percent  of  the  scientists  in  one  grade  have 
a  publication  score  per  year  as  high  as  or 
higher  than  50  percent  of  the  scientists  in  the 
next  higher  grade  and  25  percent  of  the  scien- 
tists in  the  second  higher  grade. 

10  percent  of  the  scientists  in  one  grade  have 
a  publication  score  per  year  as  high  as  or 
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Figure  4. — Cumulative  percentage  of  scientists,  grades  GS-7  to  GS-15,  having  a  given  publication 

score  or  less  per  year. 


Table  13. — Number  oj  scientists  having  a  given  publication  score  per  year,  total  and  by  grade 


Publication 
score  per  year 


0  to  5 

5+  to  10... 
10+  to  15.. 
15+  to  20.. 
20+  to  25. . 
25+  to30__ 
30+  to  35-. 
35+  to  40.- 
40+  to  45-. 
45+  to  50.. 


50  + 
55  + 
60  + 
65  + 

70  + 

75  + 
80  + 

s.-,  ■ 
'.HI  - 

95  + 


to  55.. 
to  60.. 
to  65. 
to  70._ 
to  75-. 
to  80.- 
to  85.. 
to  90.. 
to  95.. 
to  100. 


107 
18 
9 
7 
3 
3 
1 


128 

76 

45 

28 

13 

15 

S 

6 

3 

•> 


Grade 


11 


115 

133 

110 

84 

46 

34 

22 

26 

10 

9 

5 

7 
7 
3 

1 


12 

13 

44 

12 

88 

37 

113 

52 

118 

73 

100 

65 

79 

61 

69 

55 

48 

37 

45 

38 

21 

43 

15 

23 

17 

23 

17 

10 

15 

13 

7 

10 

2 

6 

5 

5 

8 

7 

4 

8 

2 

8 

14 


16 
35 
34 
27 
25 
30 
20 
21 

7 

19 

10 

10 

4 

5 

3 

7 

6 


15 


1 
5 
10 
10 
15 
13 
20 
8 
16 
16 

15 
4 
8 
6 
2 
5 
5 
4 
3 
1 


Total 


409 
365 
355 
355 
276 
232 
200 
155 
132 
113 

65 
71 
54 
48 
24 
19 
20 
29 
23 
11 


Cumulative 
total 


774 
1,  129 
1,484 

1,  760 
1,992 

2,  192 
2,347 
2,  479 

2,  592 

2,  657 
2,  728 
2,  782 
2,830 
2,  S54 
2,873 
2,993 
2,922 
J,  '.145 
2,956 


20  MISCELLANEOUS   PUBLICATION    1041,    U.S.    DEPARTMENT    OF   AGRICULTURE 

Table  13. — Number  of  scientists  having  a  given  publication  score  per  year,  total  and  by  grade — Con. 


Publication 

Grade 

Total 

Cumulative 

score  per  year 

7 

9 

11 

12 

13 

14 

15 

total 

100+  to  105 

2 

1 

4 
4 
1 
5 

4 
2 

2 
2 
1 
1 

3 

2 
3 

1 

1 

1 
2 

5 
1 
1 
1 
2 

2 
3 

13 
9 
3 
6 
6 
3 
5 
6 
4 
1 

7 
4 
2 

2,969 

105+  to  110 

2,978 

110+  to  115 

2,981 

115+  to  120 

2,987 

120+  to  125 

2,993 

125+  to  130 

2,996 

130+  to  135 

1 

3,  001 

135+  to  140 

3,007 

140+  to  145 

1 

3,  011 

145+  to  150 

3,012 

150+  to  175 

2 
2  . 

2 
2 

3,  019 

175+  to  200 

2 

3,023 

200+  to  250 

3,025 

Total.    .          ... 

150 

329 

618 

824 

615 

303 

186 

3,025 

Table  14. — Percent  oj  scientists  having  a  given  publication  score  per  year,  total  and  by  grade 


Publication 
score  per  year 


Grade 


1  1 


12 


13 


14 


15 


Total 


0  to  5 

5+  to  10-. 
10+  to  15. 
15+  to  20. 
20+  to  25. 
25+  to  30. 
30+  to  35. 
35+  to  40. 
40+  to  45. 
45+  to  50. 


50+ 
55  + 
60  + 
65  + 
70+ 
75  + 
80+ 
85  + 
90+ 
95  + 

100+ 
105  + 
110  + 
115  + 
120  + 
125  + 
130  + 
135  + 
140  + 
145  + 


to  55.. 
to  60-. 
to  65.. 
to  70_ . 
to  75_ . 
to  80_ . 
to  85-. 
to  90_. 
to  95.. 
to  100. 


to  105. 
to  110. 
to  115. 
to  120. 
to  125. 
to  130. 
to  135. 
to  140. 
to  145. 
to  150. 


150+  to  175. 
175+  to  200. 
200+  to  250. 


71.33 

12.  00 
6.  00 
4.67 
2.  00 
2.  00 
.67 


.67 


.67 


3s.  II] 

23.  10 

13.68 

8.  51 

3.95 

4.  56 

2.  43 

1.  82 

.91 

.61 


30 
30 
30 


30 

"§6" 


18.61 
21.  52 
17.80 
13.  59 
7.44 
5.  50 

3.  56 

4.  21 
1.  62 
1.46 

.81 

1.  13 

1.  13 

.49 

.  16 


.32 
.32 
.32 


5.34 

10.68 

13.71 

14.32 

12.  14 

9.  59 

8.37 

5.83 

5.46 

2.  55 

1.82 

2.  06 

2.  06 

I .  N2 

.85 

.24 

.  61 

.97 

.49 

.24 

.  24 
.  12 


12 
12" 


24 


1.95 
6.  02 
8.46 
11.  XT 
10.  57 
9.  92 
8.  94 
6.02 
6.  18 
6.  99 

3.74 
3.74 
1.63 
2.  11 
1.63 
.98 
.  81 
1.  14 
1.30 
1.30 

.  65 
.65 
.  16 
.81 
.65 
.33 
.33 
.33 
.  16 
.  16 

.  49 


0.66 
2.64 
.").  2X 
11.55 
11.22 
8.91 
8.25 
9.  90 
6.60 
6.93 

2.31 
6.27 
3.30 
3.30 

1.  32 
1.65 

.99 

2.  31 
1.98 


66 
99 
33 


.  33 


33 
66 


.66 
.66 


0.54 
2.69 
5.38 
5.38 
8.06 
6.99 
10.75 
4.30 
8.60 
8.60 

8.  06 
2.  15 
4.30 
3.23 
1.08 
2.  69 
2.69 
2.  15 
1.61 
.  54 

2.69 
.  54 
.  54 
.54 

1.08 


I  (IS 
1.61 


1.  08 
1.08" 


13.52 

12.  07 

11.  74 

11.74 

9.  12 

7.  67 

6.61 

5.  12 

4.36 

3.74 

2.  15 

2.35 

1.  79 

1.  59 

.79 

.63 

.66 

.96 

.76 

.36 

.  43 
.30 
.  10 
.20 
.20 
.  10 
.  17 
.  20 
.  13 
.03 

.23 
.  13 

.07 
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Table  15. — Cumulative  percent  oj  scientists  having  a  given  publication  score  or  less  per  year,  total  and  by 

grade 


Publication 

Grade 

Total 

score  per  year 

7 

9 

11 

12 

13 

14 

15 

0  to  5. 

71.33 
83.33 
89.33 
94.  00 
96.  00 
98.00 
98.67 

38.  91 
62.  01 
75.  69 
84  20 
88.  15 
92.  71 

95.  14 

96.  96 
97.87 
98.48 

18.61 
40.  13 
57.93 
71.52 
78.96 
84.  46 
88.  02 
92.  23 
93.85 
95.31 

96.  12 

97.  25 
98.38 

98.  87 

99.  03 

99.35 

99.67 

100.  00 

5.34 
16.  02 
29.  73 
44.  05 
56.  19 
65.78 
74  15 
79.  98 
85.  44 
87.99 

89.81 
91.87 
93.93 
95.75 

96.  60 
96.84 

97.  45 

98.  42 

98.  91 

99.  15 

99.39 
99.  51 

1.95 
7.  97 
16.  43 
28.30 
38.87 
48.  79 
57.73 
63.  75 
69.93 
76.92 

80.66 
84  40 
86.  03 

88.  14 

89.  77 
90.75 
91.56 
92.  70 
94  00 
95.30 

95.95 
96.  60 
96.76 
97.57 
98.  22 

98.  55 
98.88 

99.  21 
99.37 
99.53 

100.  00 

0.66 
3.  30 
8.  58 
20.  13 
31.35 
40.  26 
48.  51 
58.41 
65.  01 
71.94 

74  25 
80.52 
83.82 

87.  12 

88.  44 

90.  09 

91.  08 
93.  39 
95.37 

0.  54 
3.  23 
8.61 
13.  99 
22.  05 
29.  04 
39.79 
44  09 
52.  69 
61.  29 

69.35 
71.  50 
75.80 
79.03 
80.  11 
82.80 
85.  49 
87.  64 
89.  25 
89.  79 

92.  48 

93.  02 
93.  56 
94  10 
95.  18 

13.  52 

5+  to  10.. 

25.  59 

10+  to  15 

37.  33 

15+  to  20 

49.  07 

20+  to  25 

58.  19 

25+  to  30.. 

65.  86 

30+  to  35 

72.  47 

35+  to  40 

77.  59 

40+  to  45 

81.  95 

45+  to  50 

99.34 

85.  69 

50+  to  55.. 

87.  84 

55+  to  60 

98.  78 

99.  08 
99.38 

90.  19 

60+  to  65 

100.  00 

91.  98 

65+  to  70.. 

93.  57 

70+  to  75 

94  36 

75+  to  80 

99.68 

94  99 

80+  to  85.- 

95.  65 

85+  to  90_  . 

100.  00 

96.  61 

90+  to  95 

97.  37 

95+  to  100 

97.  73 

100+  to  105 

96.  03 

97.  02 
97.35 

98.  16 

105+  to  110 

98.  46 

110+  to  115 

98.  56 

115+  to  120 

98.  76 

120+  to  125 

98.  96 

125+  to  130 

97.68 

99.  06 

130+  to  135 

99.63 

96.  26 

97.87 

99.  23 

135+  to  140 

98.01 
98.  67 

99.  43 

140+  to  145 

99.75 

99.  56 

145+  to  150 

99.  59 

150+  to  175 

99.33 
100.  00 

98.95 

99.  82 

175+  to  200 

100.  00 

99.  95 

200+  to  250 

100.  00 

100.  00 

Table  16. — Publication  score  per  year  to  Jan.  1,  1965,  not  exceeded  by  the  given 
percentage  oj  scientists,  by  grade  of  scientist 


Percentage  of 

Grade 

scientists 

7 

9 

11 

12 

13 

14 

15 

97 

26 

40 

55 

77 

115 

110 

138 

90 

15 

26 

36 

^53 

75 

80 

97 

75 

6 

15 

^22 

35 

M8 

54 

65 

50 

5 

7 

12 

\22 

31 

^35 

41 

25 

0 

5 

7 

13 

19 

23 

27 

22 
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higher  than  75  percent  of  the  scientists  in  the 
next  higher  grade,  50  percent  of  the  scientists 
in  the  second  higher  grade,  and  25  percent  of 
the  scientists  in  the  third  higher  grade. 

3  percent  of  the  scientists  in  one  grade  have 
a  publication  score  per  year  as  high  as  or 
higher  than  90  percent  of  the  scientists  in  the 
next  higher  grade,  75  percent  of  those  in  the 
second  higher  grade,  50  percent  of  those  in  the 
third  higher  grade,  and  25  percent  of  those 
in  the  fourth  higher  grade. 

The  mean  number  of  publications  and  the  mean 
publication  score  of  scientists  by  age  of  scientist  is 
given  in  table  17.  While  there  was  wide  variation 
among  individual  scientists  in  number  of  publica- 
tions and  publication  scores  as  shown  above,  the 
means  for  each  age  group  show  a  somewhat  regu- 
lar increase  with  age. 

In  table  17  and  tables  to  follow,  the  numbers  of 
scientists  used  in  calculating  means  of  publication 
scores  differ  from  the  numbers  used  in  calculating 
means  of  publication  numbers.  Responses  from 
scientists  reporting  publication  score  per  year  in 
excess  of  those  shown  in  tabulation  below  were 
not  used  in  calculating  mean  publication  scores. 
They  were  not  used  because  a  few  very  high  re- 
sponses made  the  means  unrepresentative  of  the 
group.  For  example,  if  all  responses  for  grade  7 
were  used  in  calculating  the  mean  publication 
score  per  year,  the  mean  would  be  4.9,  whereas  us- 
ing 124  responses  the  mean  was  1.8.  The  effect  of 
leaving  out  a  few  high  responses  was  to  make  the 
mean  and  median  responses  about  the  same. 

The  following  are  the  maximum  publication 
scores  per  year  to  January  1,  1965,  used  in  cal- 
culating the  means : 

Grade  of  scientist  Maximum  score 

7 10 

9 20 

11 40 

12 60 

13 80 

14 100 

15 120 


Table  17. — Mean  number  of  publications  and  mean 
publication  score  to  Jan.  1,  1965,  by  age  of 
scientist 


Age 

June  30, 

1965 


22 
23 
24 
25 
26 
27 
28 
29 

30 
31 
32 
33 
34 
35 
36 
37 
38 
39 

40 

41 
42 
43 
H 
45 
46 
47 
48 
19 

50 
51 
52 
53 
54 
55 
56 
57 
58 
59 

en 

(il 

62 

<;:; 

64 

cr, 

66 

67. 

lis 

f.'i 


Number 
of  scien- 
tists 


2 
4 
16 
20 
33 
49 
57 
63 

82 
61 
88 

106 

113 
88 

119 
95 

112 
97 

109 

100 

102 

124 

136 

99 

79 

82 

70 

74 

76 
83 
58 
52 
48 
57 
50 
54 
46 
61 

54 
42 
31 

2  s 
29 
24 
19 
22 
8 
6 


Mean 
number 
of  publi- 
cations 


0 
0 
1.  1 

1.  9 

2.  2 
1.7 
3.0 
4.9 

5.  2 
5.7 
6.7 
7.0 
8.  1 
8.9 
11.  2 
13.  3 

13.  2 
16.  8 

14.  4 
14.  8 

18.  5 
21.  3 
20.  5 

19.  2 

24.  3 

25.  2 

32.  7 
29.9 

27.  2 

33.  2 

34.  6 

37.  6 
33.  7 
45.  9 
44.0 
36.  8 
42.  9 

38.  1 

39.8 

44.  1 

51.  4 

45.  1 

52.  5 
41.8 
36.  7 
62.  6 
51.  8 
65.2 


Number 
of  scien- 
tists 


2 
4 
14 
14 
29 
46 
49 
58 

73 
50 
73 
90 

101 
77 

104 
88 
99 
84 

97 
93 
93 
113 
126 
93 
74 
77 
59 
70 

72 
75 
57 
47 
44 
54 
46 
52 
44 
60 

50 
42 
31 
27 
25 
22 
17 
22 


Mean 
publica- 
tion score 


0 
0 
5 
28 
44 
24 
40 
76 

72 
81 
91 
106 
148 
149 
175 
220 
244 
269 

250 
266 
338 
386 
342 
422 
408 
478 
556 
540 

569 
558 
558 
653 
706 
800 
889 
843 
780 
737 

851 
852 
951 
878 
975 
836 
713 
808 
976 
1,  108 


USE    OF    QUALITY    AND    QUANTITY    OF    PUBLICATION    FOR    EVALUATING    SCIENTISTS 


23 


Publication  Records  of  ARS  Scientists  as  Related  to  Grade,  Degree,  Position  Title,  Length 
of  Experience,  Work  Location,  and  Place  of  Previous  Employment 


Grade 

Publication  records  of  ARS  scientists  by  grade 
arc  given  in  table  18.  As  expected,  there  is  a  sig- 
nificant increase  from  grade  to  grade  with  one 
exception  regardless  of  the  publication  measure 
used.  Mean  publication  credit  per  year  is  not  sig- 
nificantly different  for  grades  13  and  14.  The 
scientists  in  grade  15  produce  about  twice  as  many 
publications  per  year  and  have  a  publication  score 
per  year  about  twice  as  big  as  the  scientists  in 
grade  12. 

Degree 

A  comparison  of  publication  records  of  ARS 
scientists  having  different  degrees  is  shown  in  table 
19.  Those  scientists  having  doctor's  degrees  have 
1.35  times  the  publications  per  year,  1.55  times  the 
publication  credit  per  year,  and  1.53  times  the 
publication  score  per  year  as  those  having  master's 
degrees.    For  all  respondents,  the  mean  number  of 


publications  per  year  was  1.3  for  all  the  years  they 
have  been  publishing. 

Position  Title 

The  publication  records  of  ARS  scientists  by 
position  titles  on  January  1,  1965,  are  given  in 
table  20.  To  find  that  scientists  having  the  posi- 
tion title  "Research  Scientist"  had  the  lowest  pub- 
lications per  year  was  somewhat  of  a  surprise. 
Looking  into  this,  it  was  soon  apparent  what  the 
reasons  were.  All  employees  in  grade  7  and  most 
of  those  in  grades  9  and  11  were  listed  as  research 
scientists,  whereas  the  grades  of  people  with  other 
position  titles  were  nearly  all  grade  12  and  above, 
as  shown  in  table  21.  For  this  reason,  table  22 
was  prepared  using  responses  for  only  those  sci- 
entists who  were  in  grades  12  and  above.  Here, 
the  record  of  "Research  Scientists"  looks  some- 
what better.  Scientists  having  the  position  titles 
"Investigations  Leader"  and  "Laboratory  Chief" 


Table  18. — Means  of  publication  records  oj  ARS  scientists  to  Jan.  1,  1965,  and  number  of  respondents 

used  in  calculating  the  means,  by  grade  of  scientist  1 


Item 


Grade 


11 


12 


13 


14 


15 


Number  of  respondents. 

Publications: 

Total,  mean 

Per  year,  mean 

Publication  credit: 

Total,  mean 

Per  year,  mean 


Number  of  respondents  2. 

Publication  score: 

Total,  mean 

Per  year,  mean 


149 

1.960 
.327 

.638 
.  114 

124 

13.  202 
1.814 


328 

4.896 
.651 

2.  096 
.288 

276 

52.  116 
6.657 


614 

9.  836 
.  914 

5.  047 
.485 

566 

149.  816 
13.  588 


820 

16.310 
1.315 

9.995 
.813 

752 

293.  914 
23.  080 


614 

27.  399 
1.695 

16.  901 
1.  140 


303 

44.  584 
2.  004 

26.  361 
1.227 


556 

516.  763 
31.  559 


288 

861.  726 
38.  150 


185 

63.  151 
2.443 

36.  936 
1.474 

175 

1,  187.  269 
45.  714 


1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  less  than  the  1-percent  level. 
Any  2  means  underscored  by  the  same  line  are  not  signifi- 
cantly different  at  the  5-percent  level. 


2  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p.  22. 
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have  significantly  more  publications  per  year  and 
significantly  higher  publication  scores  per  year 
than  scientists  having  other  position  titles. 

Table  23  compares  the  publication  records  of 
scientists  in  grades  12  through  15  who  have  the 
position  title  "Research  Scientist/'  Those  in 
grade  15  have  records  more  than  double  those  of 
scientists  in  grade  12. 


Length  of  Experience 

Scientists'  publication  records  are  compared  in 
table  24  in  relation  to  the  length  of  time  they  have 
been  publishing.  Scientists  who  have  been  pub- 
lishing 21  or  more  years  have  significantly  better 
records  than  those  who  have  published  20  years 
or  less.  There  is  no  indication  that  a  scientist 
reaches  a  maximum  rate  of  publication  in  early 


Table  19. — Means  J  of  publication  records  to  Jan.  1,  1965,  oj  AJRS  scientists  having  specified  degrees  on 
Jan.  1,  1965,  and  number  oj  respondents  used  in  calculating  the  means 


Item 

Ph.  D., 
Sc.  D. 

M.S.,  M.A., 
D.V.M. 

B.S.,  B.A. 

No  degree 

All 
respondents 

Number  of  respondents 

Number  of  publications  per  year,  mean  l 

1,346 
1.  680 

1.  097 

1,217 
31.  367 

841 
1.  244 

.  707 

768 
20.  439 

802 
0.  889 

39 
0.  746 

3,028 
1.335 

Publication  credit  per  year,  mean  ' . 

.  451 

.334 

.  808 

Number  of  respondents  2 
Publication  score  per  year,  mean  ! 

734 
13.  847 

34 
12.  206 

2,753 
23.  408 

1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  1-percent  level. 


2  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p.  22. 


Table  20. — Means  l  oj  publication  records  to  Jan.  1,  1965,  oj  ARS  scientists  by  position  title  on  Jan.  1, 
1965,  and  number  oj  respondents  used  in  calculating  the  means 


Item 

Research 
scientist 

Field 
station 
superin- 
tendent 

Project 
leader 

Division 
director 

Labora- 
tory 
director 

Branch 
chief 

Investi- 
gations 
leader 

Labora- 
tory 
chief 

Number  of  respondents. . 
Number  of  publications 
per  year,  mean..  _ 

2,069 
1.  171 

49 
1.385 

360 
1.395 

70 
1.558 

26 
1.615 

73 
1.656 

326 

2.008 

1.247 

54 
2.405 

.694 

Publication  credit  per 
year,  mean 

.  761 

.899 

.  934 

1.095 

1.  108 

1.239 

1,847 
20.  149 

47 
19.  032 

328 
24.  139 

70 
28.  319 

Number  of  respondents  2_ 
Publication  score  per 
year,  mean. 

26 
3  30.  438 

71 
27.  600 

312 
3  37.  253 

51 
40.  494 

1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  5  percent  level,  except  as 
noted  in  footnote  3. 


2  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p. 22. 

3  30.438  is  not  significantly  different  from  37.253. 
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years  and  then  has  a  reduced  rate  in  later  years 
as  some  have  suggested  {2,3,6).  On  the  cont  ra  i ■  \ , 
the  data  in  table  24  confirm  the  findings  of  Stewart 
and  Sparks  (8)  that  productivity  increases  with 
age.  The  publication  records  of  the  four  sci- 
entists producing  the  highest  number  of  publica- 
tions per  year  in  grades  12,  13,  14,  and  15  were 
examined  in  more  detail. 


In  grade  12  the  range  in  publications  per  year 
went  from  0  to  6.6.  The  scientist  who  produced 
6.6  publications  per  year  had  a  total  of  236  publi- 
cations. He  has  been  publishing  36  years.  Dur- 
ing the  first  26  years  he  published  153  papers,  or 
5.9  publications  per  year.  During  the  last  10 
years  he  published  83  papers,  a  per-year  rate  of  8.3. 


Table  21. — Number  of  ARS  scientists  having  a  given  position  title,  by  grade 


Grade 

Research 
scientist 

Project 
leader 

Investi- 
gations 
leader 

Labora- 
tory 
director 

Field  sta- 
tion super- 
intendent 

Labora- 
tory 
chief  ' 

Branch 
chief  ' 

Division 
director  ' 

7   _           

149 
310 
540 
640 
330 
75 
24 

9   - 

15 

58 

128 

128 

31 

2 

3 

28 

99 

136 

57 

1 
7 

16 
21 

4 

11.  .. 

1 
1 
8 
10 
6 

1 
1 
2 
13 
35 
2 

1 

1 
14 
28 
29 

2 

12... 

5 

13...  

12 

14.. 

6 

15.. 

34 

16.- 

9 

17.. 

1 

2 

18-. 

Total 

1 

2,069 

360 

326 

26 

49 

54 

73 

70 

All  scientists  in  the  Office  of  Chief  or  Director. 


Table  22. — Means  of  publication  records  to  Jan.  1,  1965,  oj  ARS  scientists  grade  12  and  above,  by  position 

title,  and  number  oj  respondents  used  in  calculating  means 


Item 

Field 

station 
superin- 
tendent 

Project 
leader 

Branch 
chief 

Division 
director 

Labora- 
tory 
director 

Research 
scientist 

Investi- 
gations 
leader 

Labora- 
tory 
chief 

Number  of  respondents.. 
Number  of  publications 
per  vear,  mean  l 

49 
1.385 

287 
1.562 

73 
1.656 

70 
1.558 

26 
1.615 

1,070 
1.579 

326 
2.008 

312 
3  37.  253 

54 
2.  405 

Number  of  respondents.  _ 
Publication  score  per 
vear,  mean  - 

47 
19.  032 

266 
26.  725 

71 
27.  600 

70 
28.  319 

26 
3  30.  438 

963 
29.  572 

51 
3  40.  494 

1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  5-percent  level. 

2  Any  2  means  not  underscored  by  the  same  line  are 
significantly    different   at    the    1-percent   level   except   as 


noted  in  footnote  3. 

3  30.438  is  not  significantly  different  from  37.253  at  the 
5-percent  level  and  is  significantly  different  from  40.494 
at  the  5-percent  level. 


26 


MISCELLANEOUS   PUBLICATION    1041,    U.S.    DEPARTMENT    OF    AGRICULTURE 


Table  23. — Means  of  publication  records  to  Jan.  1. 
1965,  of  ARS  "Research  scientists,"  by  grade, 
and  number  of  respondents  used  in  calculating  the 


means 

Item 

Grade 

12 

13 

14 

15 

Number  of  respondents.. 
Number  of  publications 
per  year,  mean  '_ 

640 

1.330 

588 

24.  015 

330 

1.  782 

292 

34.  776 

75 
2.347 

24 
2.  942 

Number  of  respondents.. 
Publication  score  per 
year,  mean  l 

61 
46.  766 

21 
60.  086 

1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  1-percent  level.  46.766  and 
60.086  are  significantly  different  at  the  5-percent  level. 


In  grade  13  the  range  in  publications  per  year 
went  from  0  to  6.7.  The  scientist  who  produced 
6.7  publications  per  year  had  a  total  of  209  pub- 
lications. He  has  been  publishing  31  years.  Dur- 
ing the  first  21  years  he  published  130  papers  with 
a  per-year  rate  of  6.2.  During  the  last  10  years 
he  published  79  papers  with  a  per-year  rate  of  7.9. 

In  grade  14  the  range  in  publications  per  year 
went  from  0.9  to  10.4.  The  scientist  who  pro- 
duced 10.4  publications  per  year  had  a  total  of  187 
publications.  He  has  been  publishing  18  years. 
During  the  first  8  years  he  produced  60  publica- 
tions, a  per-year  rate  of  7.5.  During  the  last  10 
years  he  published  127  papers,  a  per-year  rate  of 
12.7. 

In  grade  15  the  range  in  publications  per  year 
went  from  1.1  to  9.6.  The  scientist  producing  9.6 
publications  per  year  had  a  total  of  278  publica- 


tions. He  has  been  publishing  29  years.  During 
the  first  19  years  he  published  135  papers,  a  per- 
year  rate  of  7.1.  During  the  last  10  years  he  pub- 
lished 143  papers,  a  per-year  rate  of  14.3. 

Work  Location 

The  publication  records  of  scientists  located  on 
or  near  (10  miles)  a  college  campus  are  com- 
pared with  scientists  located  more  than  10  miles 
from  a  college  campus  in  table  25.  The  publica- 
tion records  were  not  significantly  different. 

ARS  scientists  working  in  cooperator  facilities 
produced  more  publications  per  year  than  those 
working  in  Federal  facilities,  as  shown  in  table  26. 

Table  24. — Means  of  publication  records  to  Jan.  1, 
1965,  of  ARS  scientists,  by  length  of  time  they 
have  been  publishing,  and  number  of  respondents 
used  in  calculating  the  means 


Item 

31  or 
more 
years 

21  to 

30 
years 

11  to 
20 

years 

0  to 

10 

years 

Number  of  respondents.  . 
Number  of  publications 
per  year,  mean  ' 

247 
1.684 

407 
2 1.  675 

904 
2 1.  507 

.  943 

1,460 
1.089 

Publication  credit  per 
year,  mean ' 

1.069 

.  985 

.  636 

Number  of  respondents  3_ 
Publication  score  per 
year,  mean  •    .    . 

234 
31.  586 

385 
31.  187 

835 
26.  822 

1,289 

17.  576 

1  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  1-percent  level,  except  as  noted 
in  footnote  2. 

2  1.507  is  significantly  different  from  1.675  at  the  5-per- 
cent level. 

3  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p.  22. 


Table  25. — Publication  records  to  Jan.  1,  1965,  of  ARS  scientists  located  on  or  near  (10  miles)  a  college 

campus  and  those  more  than  10  miles  from  a  campus 


Item 


Mean 


On  or  near 
campus 


Off 
campus 


Difference 
in  means 


Number  of  respondents 


On  or  near 
campus 


Off 

campus 


Publications  per  year 

Publication  credit  per  year 
Publication  score  per  year. 


1.325 

.  791 

23.  495 


1.  411 

.875 

23.  273 


1  0.  086 
1  .084 
1  .222 


2,458 
2,  458 
2,249 


544 
544 
481 


Not  significantly  different  at  5-percent  level. 

Certain  high  responses  were  not  used  in  calculations  relating  to  publication  scores.     Sec  p. 
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The  publication  scores  per  year  were  not  signif- 
icantly different. 

When  both  ownership  of  facilities  and  work  lo- 
cation were  considered,  as  in  table  27,  it  turns  out 
that  scientists  in  cooperator  facilities  located  off 
campus  produced  the  greatest  number  of  publica- 
tions per  year,  but  had  the  lowest  publication  score 
per  year.  However,  the  publication  scores  per 
year  were  not  .significantly  different  for  any 
comparison. 

While  not  many  means  in  tables  25,  26,  and  27 
were  significantly  different,  those  that  were  tend 


to  favor  scientists  working  in  cooperator  facilities 
and  those  located  off  campus.  It  turns  out  that 
these  differences  are  not  due  to  work  location  or 
ownership  of  facilities  in  which  the  scientists 
work,  but  rather  to  the  differences  in  average 
degrees  of  the  scientists  working  in  the  different 
situations,  as  shown  in  tables  28,  29,  and  30.  It 
must  be  concluded  that  scientists  with  the  same 
training  and  experience  produce  as  well  in  one 
situation  as  another.  This  is  contrary  to  the  gen- 
eral belief  that  working  on  a  college  campus  leads 
to  higher  productivity. 


Table  26.- — Means  of  publication  records  to  Jan.  1,  1965,  oj  ARS  scientists  working  in  Federal  (F)  facilities 
and  cooperator  (C)  facilities,  difference  in  means,  and  number  of  respondents  used  in  calculating  the 
means 


Item 


Mean 


Difference 
in  means 


Number  of  respondents 


Publications  per  year 

Publication  credit  per  year 

Publication  score  per  year 

♦♦Significantly  different  at  1-percent  level. 
1  Not  significantly  different  at  5-percent  level 


1.  302 

.  754 

23.  227 


1.  485 

.952 

24.  367 


**0.  183 
**.  198 
1  1.  140 


2,374 

2,374 

2  2,  172 


640 

640 

2  568 


2  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p.  22. 


Table  27. — Range  and  mean  of  publication  records  to  Jan.  1,  1965,  of  ARS  scientists  working  in  Federal 
and  cooperator  facilities,  on  and  off  campus,  and  number  of  respondents 


Federal  facilities 

Cooperator  facilities 

Item 

On  or  near 
campus  ' 

Off 
campus 

Off 
campus 

On  or  near 
campus  ' 

Number  of  respondents  ...         . 

1,955 

10-0 

1.287 

407 

7-0 
1.374 

137 

9.6-0 
1.523 

503 

Number  of  publications  per  year: 
Range .    . 

9.  9-0 

Mean2. _ 

1.475 

(3) 
.749 

1,800 

117-0 
23.  142 

Publication  credit  per  year: 

Range .              ... 

(3) 
.853 

7.  1-0 
.939 

7.  6-0 

Mean  2. 

.955 

362 

110-0 
23.  586 

Number  of  respondents  4' 

119 

80-0 
22.  321 

449 

Publication  score  per  year: 

Range .    . 

88-0 

Mean  2. 

24.  909 

1  Within  10  miles. 

2  Any  2  means  not  underscored  by  the  same  line  are 
significantly  different  at  the  5-perccnt  level. 


3  Not  available. 

4  Certain  high  responses  were  not  used  in  calculations 
relating  to  publication  scores.     See  p.  22. 
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Publication  Records  Before  and  After  Entering  ARS 

Those  scientists  who  had  been  continuously  em- 
ployed by  ARS  since  January  1,  1955,  and  who 
had  previous  full-time  employment  in  research 
or  research  and  teaching  improved  their  publica- 
tion records  after  joining  ARS  without  regard  to 
their  place  of  previous  employment,  as  shown  in 
table  31.  However,  when  one  takes  account  of  dif- 
ferences in  publication  records  as  related  to  length 
of  experience  for  all  ARS  scientists  in  table  24 
and  for  those  continuously  employed  since  Jan- 
uary 1,  1955,  in  table  32,  it  is  apparent  that  most 
of  the  differences  shown  in  table  31  are  due  to  the 
additional  years  of  experience. 


Table  28. — Average  degree,  average  grade,  and  aver- 
age length  of  experience  of  ARS  scientists  working 
on  and  off  campus 


Item 

On 
campus  1 

Off 

campus 

Average  degree  2    _ 

3.  14 
11.85 

13.25 

3.  21 

Average  grade  _ 

11.  81 

Average  length  of  experience 
(years)                                      -   . 

12.  55 

1  On  or  near  (10  miles). 

2  Rating: 

Doctors — 4; 
Masters — 3 ; 
Bachelors — 2 ; 
No  degree — 1. 


Table  29. — Average  degree,  average  grade,  and 
average  length  of  experience  of  ARS '  scientists 
working  in  Federal  and  cooperator  facilities 


Item 


Average  degree  ' 

Average  grade 

Average  length  of  experience 
(years) 


Federal 
facilities 


3.08 
11.83 

13.36 


1  Rating: 

Doctors — 4; 
Masters — 3 ; 
Bachelors — 2 ; 
No  degree — 1. 

Table  30. — Average  degree,  average  grade,  and 
average  length  of  experience  of  ARS  scientists 
working  in  Federal  and  cooperator  facilities,  on 
and  off  campus 


Item 

Federal  facilities 

Cooperator 
facilities 

On 
campus  ' 

Off 
campus 

On 

campus  J 

Off 
campus 

Average  degree  2    . 
Average  grade 
Average  length  of 
experience   (years) . 

3.07 

11.  84 

13.  50 

3.  13 

11.  80 

12.65 

3.41 

11.  87 

12.  27 

3.45 
11.83 

12.25 

1  On  or  near  (10  miles). 

2  Rating: 

Doctors — 4; 
Masters — 3; 
Bachelors — 2  ; 
No  degree — 1. 


Publications  and  Promotions 


A  majcr  objective  of  this  study  was  to  find  what 
relation,  if  any,  there  was  between  publications 
(number  and  quality)  and  promotions.  Before 
the  reorganization  of  ARS  in  1954,  scientists  were 
in  different  bureaus  having  somewhat  varying  per- 
sonnel policies.  Since  1954,  except  for  two  divi- 
sions which  joined  ARS  in  1964,  scientists  in  ARS 
have  been  under  uniform  personnel  policies. 
These  policies  have  changed  since  1954,  but  they 
have  changed  uniformly  for  all.  Consequently, 
the  period  from  January  1, 1955  to  January  1, 1965 
was  selected  as  the  best  period  to  study  the  relation 
between  publications  and  promotions.  Only  the 
responses  from  the  1,327  scientists  continuously 


employed  by  ARS  from  January  1,  1955  to  Janu- 
ary 1,  1965  were  considered  in  analyzing  the  rela- 
tion between  publications  and  promotions. 

Correlation  Between  Promotions  and  Publications 
by  Grade  of  Scientist 

Correlation  coefficients  between  the  number  of 
grade  changes  from  January  1,  1955  and  18  pub- 
lication measures  by  grade  of  scientists  on  Janu- 
ary 1,  1955  are  shown  in  table  33.  The  total  num- 
ber of  publications  before  January  1,  1955,  was 
not  positively  correlated  with  promotions,  but 
number  per  year  before  January   1,   1955   was 
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Table  31.— Publication  record  before  and  after  entering  ARS  of  scientists  continuously  employed  by  ARS 
since  Jan.  1,  1955,  who  had  previous  full-time  employment  in  research  or  research  and  teaching  by  place 
of  previous  employment 


Place  of  previous  employment,  number  of  scientists,  and  publication  record 


University,  297  scientists 

Publications  per  year 

Publication  credit  per  year .. 

Publication  score  per  year 

Industry,  66  scientists 

Publications  per  year 

Publication  credit  per  year 

Publication  score  per  year 

Federal  Government,  72  scientists 

Publications  per  year 

Publication  credit  per  year 

Publication  score  per  year 

Other,1  85  scientists 

Publications  per  year 

Publication  credit  per  year 

Publication  score  per  year 

Total,  520  scientists 

Publications  per  year 

Publication  credit  per  year 

Publication  score  per  year 


Mean 


Before 


].  248 

.  805 

26.  269 


.  609 

.339 

10.  575 


.  622 

.  406 

11.  976 


1.  086 

.677 

21.  990 


.  891 
.  556 

17.  703 


After 


1.  895 

1.  109 

35.  402 


1.  475 

.  773 

27.  343 


1.333 

.  822 
22.  547 


1.781 

1.  068 

33.  381 


1.  621 

.  943 

29.  668 


Difference 
in  means 


**0.  647 

**.  304 

**9.  133 


**.  866 

**.  434 

**16.  768 


**.  711 

**.  416 

**10.  571 


**.  695 

**.  391 

**11.  391 


**.  730 

**.  387 

**11.  965 


**Significantly  different  at  1-percent  level. 

1  Usually  previously  employed  by  two  or  more  of  the  other  three  categories. 


correlated  with  promotions  in  the  following  10 
years.  Total  publications  between  January  1, 
1955  and  January  1,  1965,  total  publications  to 
January  1,  1965,  publications  per  year  between 
January  1, 1955,  and  January  1, 1965,  and  publica- 
tions per  year  to  January  1,  1965  were  all  highly 
correlated  with  promotions  from  January  1,  1955 
to  January  1, 1965. 

The  same  relationships  hold  for  publication 
credit  and  publication  score  as  those  given  above 
for  number  of  publications. 

For  the  two  time  periods  January  1,  1955  to 
January  1,  1965  and  total  to  January  1,  1965, 
which  show  publications  to  be  correlated  with  pro- 
motions, the  per-year  values  were  more  highly 
correlated  with  promotions  than  total  values  were 
in  31  out  of  36  comparisons  in  grades  5  through  13. 
Publication  score  per  year  was  only  slightly  better 
correlated  with  promotions  than  publications  per 
year — 7  out  of  12  comparisons  in  grades  5  through 
13.     Publication  score  per  year  was  more  highly 


correlated  with  promotions  than  publication  credit 
per  year  in  9  out  of  12  comparisons  in  grades  5 


through  12. 


Table  32. — Mean  and  range  of  three  publication 
measures  to  Jan.  1,  1955,  and  between  Jan.  1, 
1955,  and  Jan.  1,  1965,  for  the  1,327  scientists 
employed  continuously  by  ARS  since  Jan.  1,  1955 


Item 

Publi- 
cations 
per 
year 

Publi- 
cation 
credit 
per 
year 

Publi- 
cation 
score 
per 
year 

To  Jan.  1,  1955 
Mean 

1.08 
7.  50 
0 

1.81 
14.30 

0 

0.64 
6.  10 
0 

1.04 
11.  90 

0 

19.  75 

Maximum ..    

143.  00 

Minimum.          .              

0 

Jan.  1,  1955,  to  Jan.  1,  1965 
Mean - 

32.43 

Maximum..    

164.  60 

Minimum 

0 

30 
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Table  33. — Correlation  coefficients  between  number  of  grade  changes  from  Jan.  1,  1955,  to  Jan.  1, 
and  various  ■publication  measures,  by  grade  of  scientists  on  Jan.  1,  1955 


1965, 


Publication  measure 


Total  publications 

1.  To  Jan.  1,  1955 

2.  Jan.  1,  1955  to  Jan.  1,  1965. 

3.  To  Jan.  1,  1965 


Publications  per  year 

4.  To  Jan.  1,  1955 

5.  Jan.  1,  1955  to  Jan.  1,  1965.. 

6.  To  Jan.  1,  1965 


Publication  credit 

7.  To  Jan.  1,  1955 

8.  Jan.  1,  1955  to  Jan.  1,  1965. 

9.  To  Jan.  1,  1965 


Publication  credit  per  year 

10.  To  Jan.  1,  1955 

11.  Jan.  1,  1955  to  Jan.  1,  1965.. .. 

12.  To  Jan.  1,  1965 


Publication  score 

13.  To  Jan.  1,  1955 

14.  Jan.  1,  1955  to  Jan.  1,  1965. 

15.  To  Jan.  1,  1965 


Publication  score  per  year 

16.  To  Jan.  1,  1955 

17.  Jan.  1,  1955  to  Jan.  1,  1965 

18.  To  Jan.  1,  1965 


0.  138 
**.  534 
**.  551 


*.  269 

**.  557 
**.  623 


.  155 

**.  480 
**.  501 


.215 
**.  492 

**.  575 


.  197 
**.  640 

**.  593 


*.  297 
**.  668 

**.  667 


Grade,  Jan.  1,  1955 


-0.  019 
**.  544 

**.  355 


**.  204 
**.  455 
**.  443 


.072 
**.  479 

**.  420 


»*.  227 

**.  479 
**.  229 


.017 
**.  442 

**.  355 


*  161 
**.  442 

**.  428 


-0.057 
**.  382 
**.  251 


**.  189 
**.  387 

**.  412 


-.036 
**.  312 

**.  210 


**.  183 
**.  319 
**.  343 


-.017 
**.  407 

**.  196 


**.  240 
**.  412 
**.  437 


11 


-0.082 
**.  372 
**.  178 


*.  134 
**.  399 
**.  350 


-.083 

**.  261 

*.  133 


.  Ill 
**.  249 

**.  227 


.  002 
**.  465 
**.  283 


**.  181 
**.  466 
**.  426 


12 


0.  Ill 
**.  325 
**.  255 


**.  232 
**.  311 
**.  308 


.040 

**.  218 

*  162 


*.  128 
**.  221 
**.  195 


**.  185 

*.  147 

**.  276 


**.  311 
**.  310 

**.  361 


13 

14 

0.  126 
**.  286 

*.  234 

0.084 
.068 
.088 

*.  217 
**.  297 
**.  279 

.020 
.068 
.079 

.098 

**.  270 

*.  196 

-.084 

.078 

-.038 

*.  198 
**.  266 

*.  247 

-.  101 

.  075 
-.038 

.  101 

*.  215 

.095 

.  116 
.  263 
.  192 

.  177 
*  220 
*.  219 

.  160 

.264 
.250 

All 
grades  ' 


**-0.  154 
**.  190 
-.005 


.007 
**.  200 
**.  195 


**— .  157 
**.  142 
-.031 


-.010 
**.  143 

**.  132 


**— .  121 

**.  087 
.008 


.023 

**.  221 
**.  212 


♦Significant  at  5-percent  level 0.  264 

♦♦Significant  at  1-percent  level .  344 

1  Including  grade  15. 


0.  143 
.  188 


0.  107 
.  141 


0.  117 
.  153 


0.  128 
.  170 


0.  194 

.  254 


0.384 
.496 


0.056 
.073 


As  would  be  expected,  the  highest  correlations 
between  publication  score  per  year  and  promotions 
were  obtained  for  those  employees  who  were  in 
grade  5  on  January  1, 1955.  About  equal  correla- 
tions were  found  for  those  in  grades  7,  9,  and  11 
on  January  1,  1955.  The  lowest  correlations  were 
for  those  employees  who  were  in  grade  12  and 
above  on  January  1, 1955.  None  of  the  correlation 
coefficients  are  significant  for  employees  who  were 
in  grade  14  on  January  1, 1955. 

Correlation  Between  Promotions  and  Publications 
by  Grade  of  Scientist  and  by  Degree  Held 

Scientists  were  separated  by  degree  held  on  Jan- 
uary 1, 1955  as  well  as  by  grade  on  January  1, 1955 
to  determine  correlation  coefficients  between  num- 


ber of  promotions  and  publications.  The  results 
are  shown  in  table  34.  They  are  essentially  the 
same  as  those  shown  in  table  33,  where  scientists 
were  grouped  by  grade  on  January  1, 1955  without 
regard  to  degree  held.  Correlation  coefficients  be- 
tween promotions  and  publications  are  highly  sig- 
nificant for  scientists  in  grades  5  through  12  on 
January  1,  1955. 

Correlation  Between  Promotions  and  Publications 
by  Grade  and  by  Number  of  Years  a  Scientist 
Had  Published 

The  correlation  coefficients  between  publications 
and  promotions  when  scientists  were  separated  by 
length  of  experience  as  well  as  by  grade  on  Jan- 
uary 1,  1955,  as  shown  in  table  35,  are  in  essential 
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Table  34. — Correlation  coefficients  between  number  of  grade  changes  from  Jan.  1,  1955,  to  Jan.  1,  1965, 
and  4  publication  measures,  by  grade  of  scientist  and  degree  held  on  Jan.  1,  1955  ' 


Publication  measure  and  degree  on 

Grade,  Jan.  1 

,  1955 

Jan.  1,  1955 

5 

7 

9 

11 

12 

13 

14 

Publications  per  year 

Jan.  1,  1955,  to  Jan.  1,  1965: 

Doctor's.      __   

**0.  464 
**.  397 
**.  227 

**.  519 
**.  367 

**.  258 

**.  384 
**.  398 

**.  275 

**.  443 
**.  400 
**.  270 

**0.  381 
**.  379 
**.  446 

**.  328 
**.  327 
**.  388 

**.  439 
**.  425 
**.  543 

**.  383 
**.  417 

**'.  456 

**0.  378 

.  168 

*.  383 

**.  290 

**.  299 

*.  336 

**  365 
.  219 
.269 

**.  359 

**.  319 

*.  331 

*0.  280 
.  264 

0.  066 

Master's.. 

**0.  441 
**.  430 

Bachelor's.. 

**0.  595 

To  Jan.  1,  1965: 

Doctor's   

*.  257 
.  169 

.  177 

Master's..     

**.  418 
**.  410 

Bachelor's 

**.  639 

Publication  score  per  year 

Jan.  1,  1955,  to  Jan.  1,  1965: 

Doctor's ..              

.  107 
.  446 

.  306 

Master's 

**.  442 
**.  520 

Bachelor's.. 

**.  667 

To  Jan.  1,  1965: 

Doctor's   .      

.091 

**.  412 

.  384 

Master's  _.   _ 

**.  439 
**.  493 

Bachelor's..     . ._ 

**.  649 

f 

♦Significant  at  the  5-percent  level. 
"""Significant  at  the  1-percent  level. 


1  Correlation   coefficients   not   given   where   number  o 
respondents  in  a  group  was  less  than  18. 


Table  35. — Correlation  coefficients  between  number  of  grade  changes  from  Jan.  1,  1955,  to  Jan.  1,  1965, 
and  4  publication  measures,  by  grade  Jan.  1,  1955  and  number  of  years  of  publishing  to  Jan.  1,  1965  l 


Publication   measure   and  number 
of  years  of  publishing 

Grade,  Jan.  1,  1955 

5 

7 

9 

11 

12 

13 

14 

Publications  per  year 

Jan.  1,  1955,  to  Jan.  1,  1965: 

31+ 

0.  168 
**.  364 
**.  462 

**0.  283 
**.  363 

.  248 

**0.  396 
.  156 

21  to  30 

0.  422 
**.  475 
**.  371 

**0.  363 

**.  362 

*.  435 

11  to  20 

**0.  647 
**.  590 

Oto  10 

To  Jan.  1,  1965: 

31+ 

.031 

**.  325 
**.  427 

**.  283 

**.  282 

*.  305 

**.  418 
.  Ill 

21  to  30 ... 

.  342 

**.  473 
*.  279 

.  207 
**.  390 
**.  598 

11  to  20 

**.  712 
**.  561 

Oto  10 

Publication  score  per  year 

Jan.  1,  1955,  to  Jan.  1,  1965: 

31+ 

.  245 
**.  453 

**.  488 

**.  375 

**.  261 

*  332 

*.  293 
.  115 

21  to  30 

*.  579 

**.  527 

.  226 

**.  437 
**.  367 
**.  698 

11  to  20... 

**.  656 
**.  691 

Oto  10 

To  Jan.  1,  1965: 

31+ 

.087 
**.  439 

**.  450 

**.  432 
**.  279 

*.  389 

*.  330 
.  037 

21  to  30 

**.  604 

**.  524 

.  131 

**.  353 
**.  390 
**.  726 

11  to  20... 

**.  658 
**.  646 

Oto  10 

♦Significant  at  5-pcrcent  level. 
♦""Significant  at  1-perccnt  level. 


1  Correlation   coefficients   not   given   where   number   of 
respondents  in  a  group  was  less  than  18. 
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agreement  with  those  obtained  when  the  scientists 
were  separated  by  grade  only.  The  correlation 
coefficients  between  promotions  and  publications 
are  highly  significant  for  scientists  in  grades  5 
through  12  on  January  1,  1955.  However,  for 
those  scientists  in  grade  11  on  January  1,  1955 
who  had  been  publishing  21  or  more  years  at  that 


time  (31+  to  January  1,  1965),  the  correlation 
coefficients  between  publications  and  promotions 
were  not  significant.  For  those  scientists  in  grades 
12  and  13  with  21  or  more  years  of  publishing  on 
January  1,  1955  (31+  to  January  1,  1965)  the 
correlation  coefficients  between  publications  and 
promotions  were  significant. 


Net  Change  in  Quality  of  ARS  Scientists  for  Two  Time  Periods 


In  1957,  Dr.  William  Shockley  introduced  merit 
index  as  a  tool  for  studying  in  an  objective  way 
the  problem  of  quality  losses  in  civil  service  lab- 
oratories (7) .  He  defined  merit  index  for  an  indi- 
vidual for  a  given  date  as  the  fraction  of  employees 
of  his  age  that  the  individual  exceeds  in  salary. 
Thus,  the  person  having  the  top  salary  for  his  age 
will  have  an  index  of  1.0 ;  the  person  having  the 
median  salary  will  have  an  index  of  0.5;  and  the 
person  having  the  lowest  salary  will  have  an  index 
of  0.  Dr.  Shockley  analyzed  losses  of  scientists 
in  terms  of  their  distribution  in  various  merit  quar- 
tiles.  He  found  in  the  National  Bureau  of  Stand- 
ards that  the  top  quartile  had  approximately  50 
percent  more  losses  than  its  proper  share.  If  these 
losses  are  not  balanced  by  suitable  recruiting  into 
the  top  quartile,  the  quality  of  the  scientists  in  the 
laboratory  goes  down. 

The  present  study  of  quality  changes  in  AES 
modifies  Shockley's  method  of  analysis  by  assign- 
ing a  quality  value  (called  quality  factor)  to  each 
10  percentile  of  merit  indexes  as  follows : 

Quality 
Merit  index  group  factor 

1.  0.9+  to  1.0 +5 

2.  0.8+to0.9 +4 

3.  0.7+  to0.8 . +3 

4.  0.6+  to0.7 +2 

5.  0.5+  to0.6 +1 

6.  0.4+  to0.5 -1 

7.  0.3+  to0.4 -2 

8.  0.2+  to  0.3 -3 

9.  0.1+  to  0.2. -4 

10.  0.0  to  0.1... -5 

If  a  scientist  is  lost  from  the  first  merit  index 
group  (0.9+  to  1.0)  by  reason  of  resignation,  re- 
tirement, death,  etc.,  the  quality  change  is  —5 
(  —  1X5).  If  a  scientist  is  recruited  who  falls  in 
the  first  merit  index  group,  the  quality  change  is 


+  5  (  +  1X5).  If  a  scientist  is  lost  from  the  10th 
merit  index  group  (0  to  0.1),  the  quality  change  is 
+  5  (  —  IX— 5).  If  a  scientist  is  recruited  who 
falls  in  the  10th  merit  index  group,  the  quality 
change  is  —5  (  +  1X— 5).  The  net  change  in 
quality  of  ARS  scientists  during  a  given  period  is 
calculated  by  summing  the  products  of  the  net 
change  in  number  of  scientists  in  each  merit  index 
group  by  the  quality  factor  in  that  group. 

Merit  indexes  were  calculated  for  all  employees 
on  the  rolls  on  June  30,  of  1956,  1960,  and  1965. 
Those  employees  lost  between  June  30,  1956,  and 
June  30,  1960,  were  tabulated  by  their  merit  index 
on  June  30, 1956.  Those  employees  hired  between 
June  30,  1956,  and  June  30,  1960,  were  tabulated 
by  their  merit  index  on  June  30, 1960.  Similarly, 
those  employees  lost  between  June  30,  1960,  and 
June  30,  1965,  were  tabulated  by  their  merit  index 
on  June  30, 1960.  Those  employees  hired  between 
June  30,  1960,  and  June  30,  1965,  were  tabulated 
by  their  merit  index  on  June  30, 1965. 

The  reason  for  using  the  time  periods  1956  to 
1960  and  1960  to  1965  was  that  ARS  changed  the 
method  of  classifying  scientists  to  the  "man  in  the 
job"  concept  in  1959.  It  was  believed  that  the  new 
classification  procedure  would  help  in  both  reten- 
tion of  good  scientists  and  the  recruitment  of  new 
scientists. 

The  net  change  in  quality  of  ARS  scientists  by 
age  groups  and  for  all  ARS  from  June  30, 1956,  to 
June  30,  1960,  is  shown  in  table  36.  There  was  a 
small  net  decrease  in  quality  (—66)  for  all  ARS 
scientists.  This  was  made  up  by  increases  in 
quality  for  the  age  groups  24+  to  29  ( +  51),  29  + 
to  34  (  +  51),  and  54+  to  59  (  +  7),  and  decreases 
in  quality  in  all  other  age  groups. 

The  net  quality  changes  associated  with  losses  in 
the  period  June  30, 1956,  to  June  30, 1960,  and  the 
net  quality  changes  associated  with  new  recruits 
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are  shown  in  table  37.  It  is  seen  that  quality  de- 
creases associated  with  losses  occurred  in  only  two 
age  groups,  whereas  quality  decreases  associated 
with  new  recruits  occurred  in  seven  age  groups. 

The  net  change  in  quality  of  ARS  scientists  by 
age  groups  and  for  all  ARS  from  June  30,  1960, 


to  June  30,  1965,  is  shown  in  table  38.  There  was 
a  small  net  increase  in  quality  ( +87)  for  all  ARS 
scientists.  This  net  increase  is  made  up  of  a  de- 
crease in  quality  (  —  33)  in  the  age  group  44+  to 
49  and  net  increases  in  quality  for  all  other  age 
groups. 


Table  36. — Net  change  in  number  of  scientists  from  June  80,  1956,  to  June  80,  1960,  by  merit  index  group  1 
and  by  age  of  scientist  on  June  30,  1960;  also,  net  change  in  quality  of  scientists  by  age  group  and  total 
ARS  change  in  quality  2 


Age  group  June  30, 
1960 


Net  change  in  number  of  scientists,  by  merit  index  group  and 
associated  quality  factors 3 


0.9  + 
to  1.0 
(+5) 


0.8  + 
to  0.9 
(+4) 


0.7  + 
to  0.8 
(+3) 


0.6+ 
to  0.7 

(+2) 


0.5  + 
to  0.6 
(  +  D 


0.4+ 
to  0.5 
(-D 


0.3+ 
to  0.4 
(-2) 


0.2  + 
to  0.3 
(-3) 


0.1  + 
to  0.2 
(-4) 


0  to 

0.1 

(-5) 


Net 
change 
in  qual- 
ity for 

age 
group  4 


24+  to  29.. 
29+  to  34.. 
34+  to  39.. 
39+  to  44.. 
44+  to  49.. 
49+  to  54.. 
54+  to  59.. 
59+  to  64.. 
64+  to  69- 

Total 


15 

26 

14 

4 

-1 

2 

3 

-4 

-2 


14 

22 

10 

4 

3 

1 

-1 

-1 

0 


9 
22 
9 
7 
0 
2 
4 
1 
0 


10 

33 

24 

5 

1 

0 

0 

-4 

-1 


16 
25 
17 

7 
4 
1 
1 
0 
1 


9 

5 

17 

0 

1 
0 
-2 
2 
1 


5 

22 

12 

12 

0 

3 

—  1 

0 

-2 


11 

23 

15 

11 

7 

3 

1 

1 

0 


9 

24 

14 

5 

3 

-3 

2 

0 

-2 


11 

22 

16 

16 

0 

4 

2 

2 

1 


51 

51 

-20 

-83 

-21 

-2 

7 

-44 

-5 


57 


52 


54 


68 


72 


33 


51 


72 


52 


74 


■66 


1  The  merit  index  for  an  individual  for  a  given  date  is 
the  fraction  of  the  employees  of  his  age  that  the  individual 
exceeds  in  salary.  Thus,  the  person  having  the  top  salary 
for  his  age  will  have  an  index  of  1.0;  the  person  having 
the  median  salary  will  have  an  index  of  0.5;  and  the  per- 
son having  the  lowest  salary  will  have  an  index  of  0. 


2  4  utilization  research  divisions  not  included  because  of 
lack  of  payroll  data  June  30,  1956. 

3  Associated  quality  factors  are  the  figures  in  parentheses. 
*  Summation  of  products  of  number  of  scientists  times 

quality  factor  for  the  10  merit  index  groups  in  each  age 
group. 


Table  37. — Number  of  scientists  who  resigned  and  number  employed  from  June  80,  1956,  to  June  30,  1960 
by  age;  also,  quality  changes  resulting  from  losses  and  gains  1 


Age  group 
June  30,  1956 

Number 
of  scien- 
tists on 

rolls 
June  30, 

1956 

Number 
of  scien- 
tists who 
resigned 
to  June  30, 
1960 

Quality 
change 
resulting 
from  res- 
igna- 
tions 2 

Age  group 

June  30, 

1960 

N  umber 
of  scien- 
tists on 

rolls 
June  30, 

1960 

Number 
of  scien- 
tists em- 
ployed 
since 
June  30, 
1956 

Quality 

change 

resulting 

from  new 

employees 

Net  change  in 
quality  by  age 

group  on 
June  30,  1960 

20+  to  25 

25+  to  30 

30+  to  35 
35+  to  40 

40+  to  45 

45+  to  50 

50+  to  55 - 

55+  to  60 

60+  to  65 
65+  to  70 

12 

80 

196 

237 

176 

174 

187 

120 

52 

34 

5 
21 
38 
25 
17 
15 
11 
18 
19 
30 

9 

0 
15 
28 

5 

13 

10 

-18 

0 
-3 

19+  to  24.. 
24+  to  29.. 
29+  to34_. 
34+  to  39.. 
39+  to  44.. 
44+  to  49- 
49+  to  54„ 
54+  to  59.. 
59+  to  64.. 
64+  to  69.. 

17 
121 
303 
341 
307 
191 
185 
192 
116 

48 

17 

114 

245 

186 

96 

35 

28 

20 

15 

15 

0 

42 

51 

-35 

-111 

L'ti 

-15 

-3 

-26 

-5 

0 

9+  42=     51 

0+   51=     51 

15-   35= -20 

28-lll=-83 

5-   26=-21 

13-    15=    -2 

10-     3=       7 

-18-   26= -44 

0-     5=    -5 

Total— 

1,268 

199 

59 

1,821 

771 

-128 

-66 

1  4  utilization  research  divisions  not  included  because  of 
lack  of  payroll  data  June  30,  1956. 


Resignations,  death,  retirement,  etc. 


34 


MISCELLANEOUS   PUBLICATION    1041,    U.S.    DEPARTMENT    OF   AGRICULTURE 


It  would  appear  that  changing  to  the  "man  in 
the  job"  classification  had  the  desired  effects. 

The  net  quality  changes  resulting  from  losses 
and  from  new  recruits  in  the  period  June  30, 1960, 
to  June  30,  1965,  are  shown  in  table  39.  Quality 
decreases  associated  with  losses  occurred  in  three 


age  groups.  Quality  decreases  associated  with 
new  recruits  occurred  in  five  age  groups.  The  de- 
creases in  quality  associated  with  new  recruits 
was  less  in  the  period  June  30,  1960,  to  June  30, 
1965,  than  it  was  from  June  30,  1956  to  June  30, 
1960. 


Table  38. — Net  change  in  number  of  scientists  from  June  80,  1960,  to  June  SO,  1965,  by  merit  index  group  l 
and  by  age  of  scientist  on  June  30,  1965;  also,  net  change  in  quality  of  scientists  by  age  group  and  total 
ARS  change  in  quality 


Age  group  June  30, 
1965 


Net  change  in  number  of  scientists,  by  merit  index  group  and 
associated  quality  factors  2 


0.9  + 
to  1.0 

(+5) 


0.8  + 
to  0.9 
(+4) 


0.7  + 
to  0.8 
(+3) 


0.6  + 
to  0.7 
(+2) 


0.5  + 
to  0.6 
(+D 


0.4  + 
to  0.5 
(-D 


0.3  + 
to  0.4 
(-2) 


0.2  + 
to  0.3 
(-3) 


0.1  + 
to  0.2 
(-4) 


Oto 
0.1 

(-5) 


Net 
change 
in  qual- 
ity for 

age 
group  3 


24+  to  29— 
29+  to  34— 
34+  to  39.. 
39+  to  44.. 
44+  to  49- 
49+  to  54- 
54+  to  59- 
59+  to  64- 
64+  to  69- 

Total 


17 

36 

14 

5 

-4 

6 

1 

3 

-2 


22 
34 
18 

17 

4 

7 

1 

-2 

-5 


23 

8 

14 

3 

:; 

5 

3 

-4 

-4 


12 

31 

12 

9 

3 

-1 

-5 

0 

-5 


10 

8 

17 

-5 

3 

3 

0 

-4 

-5 


19 

23 

22 

11 

9 

2 

0 

-2 

-6 


17 
34 

9 
2 

1 
1 
0 
0 
-5 


18 

20 

17 

7 

-1 

5 

0 

-3 

-2 


19 

29 

10 

0 

6 

6 

-1 

-1 


17 

28 

16 

15 

3 

2 

-2 

-6 

-3 


76 


96 


51 


56 


27 


59 


61 


60 


70 


3 
14 

4 
-33 
21 
22 
36 
12 


87 


1  See  table  36,  footnote  1. 

2  Associated  quality  factors  are  the  figures  in  parentheses. 

3  Summation  of  products  of  number  of  scientists  times 


quality  factor  for  the  10  merit  index  groups  in  each  age 
group. 


Table  39. — Number  of  scientists  who  resigned  and  number  employed  from  June  80,  1960,  to  June  30,  1965, 
by  age;  also,  quality  changes  resulting  from  losses  and  gains 


Number 

Number 

Quality 

Number 

Number 

of  scien- 

of scien- 

change 

of  scien- 

of scien- 

Quality 

Net  change  in 

Age  group 

tists  on 

tists  who 

resulting 

Age  group 

tists  on 

tists  em- 

change 

quality  by  age 

June  30,  1960 

rolls 

resigned 

from  res- 

June 30, 

rolls 

ployed 

resulting 

group  on 

June  30, 

to  June  30, 

igna- 

1965 

June  30, 

since 

from  new 

June  30,  1965 

1960 

1965 

tions  * 

1965 

June  30, 
I9602 

employees 

19+  to  24 

32 

8 

-35 

19+  to  24.. 

14 

14 

0 

0 

24+  to  29 

182 

33 

-64 

24+  to  29._ 

205 

182 

43 

-35  +  43=  8 

29+  to  34.    ... 

402 

71 

35 

29+  to  34- 

433 

284 

67 

-64+67=  3 

34+  to  39 

488 

84 

39 

34+  to  39.. 

551 

220 

-21 

35-21  =  14 

39+  to  44 

442 

51 

56 

39+  to  44- 

551 

148 

-35 

39-35=  4 

44+  to  49 

310 

20 

43 

44+  to  49.. 

469 

78 

-89 

56-89=-33 

49+  to  54 

264 

23 

11 

49+  to  54.. 

346 

56 

-22 

43-22  =  21 

54+  to  59 

241 

43 

28 

54+  to  SO- 

261 

20 

11 

11  +  11  =  22 

59+  to  64 

149 

59 

30 

SO  +  to  64.. 

222 

24 

8 

28+   8  =  36 

64+  to  69 

66 

60 

-1 

64+  to  69.. 

104 

14 

-18 

30-18=12 

Total.. 

2,576 

452 

142 

3,  156 

1.  OKI 

-56 

87 

Resignations,  death,  retirement,  etc. 


2  Includes  employees  transferred  to  ARS  in  2  marketing 
divisions. 
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Merit  Index  of  Scientists  June  30,  1956,  Who  Were  Research  Supervisors  June  30,  1965 


It  is  sometimes  stated  that  research  supervisors 
in  Government  laboratories  get  their  positions  by 
default.  In  other  words,  it  is  contended  that  the 
more  able  employees  leave  Government  service  and 
employees  of  lesser  ability  thereby  get  the  promo- 
tions to  positions  which  should  have  been  filled  by 
people  of  more  ability.  It  seemed  desirable  to  test 
this  hypothesis  in  ARS.  The  results  of  the  analy- 
sis are  shown  in  table  40. 

For  those  scientists  who  have  been  continuously 
employed  by  ARS  since  January  1, 1955,  who  were 
research  supervisors  on  June  30,  1965,  the  mean 
merit  index  by  position  title  on  June  30,  1965,  was 
calculated  for  June  30,  1956.  It  was  found  that 
all  means  were  above  the  median  merit  index  for 
scientists  of  the  same  age.  It  thus  appears  that 
ARS  does  a  reasonable  job  of  selecting  research 
supervisors  from  above-average  employees  and 
that  promotions  to  these  positions  were  not  by  de- 


fault. This  is  particularly  so  for  laboratory 
chiefs  and  division  directors  whose  mean  merit 
index  9  years  before  June  30,  1965,  was  above  0.8. 

Table  40. — Merit  index  of  scientists  June  80, 1956, 
who  were  research  supervisors  on  June  80,  1965, 
by  position  title  June  30,  1965 


Position  title  June  30,  1965 

Number 
of  re- 
spond- 
ents ' 

Mean 
merit 
index 
June  30, 
1956 

Investigations  leader     

147 
18 
29 
17 
53 
21 

0.691 

Laboratory  director     -   . 

.  690 

Field  station  superintendent      

Laboratory  chief 

.  524 
.  824 

Branch  chief  2 

.  746 

Division  director 2 

.  916 

1  Only    those    employed    continuously    by    ARS    since 
Jan. 1,  1955. 

2  All  employees  in  office  of  chief  and  director. 


The  Relation  Between  Merit  Index  and  Publication  Records 


Correlation  coefficients  between  merit  index  and 
three  publication  measures  by  age  of  scientists  are 
given  in  table  41.  There  is  a  very  highly  signif- 
icant correlation  between  merit  index  and  number 
of  publications,  and  an  even  higher  correlation  be- 
tween merit  index  and  publication  scores.  In  32 
out  of  46  comparisons  publication  score  was  more 
highly  correlated  with  merit  index  than  the  num- 
ber of  publications  was.  This  shows  that  quality 
of  publications  as  well  as  quantity  of  publications 
is  considered  by  personnel  evaluation  committees 
in  classifying  scientists. 

There  was  a  much  lower  correlation  between 
merit  index  and  the  highest  rank  order  assigned 
by  scientists  to  any  of  their  publications  than  for 
the  other  two  publication  measures.  Even  so,  in 
all  but  two  age  groups  merit  index  was  correlated 
positively  with  highest  rank  order.  In  18  out  of 
46  age  groups  the  correlation  between  merit  index 
and  highest  rank  order  was  significant  at  the  1- 
percent  level. 

In  table  42  the  mean  number  of  publications 
is  given  by  age  of  scientist  and  by  merit  index. 
With  a  few  exceptions,  there  is  a  gradual  decrease 
in  the  mean  number  of  publications  going  from  the 


top  merit  index  group  to  the  bottom  index  group 
for  each  of  the  age  groupings.  Also,  with  few  ex- 
ceptions there  is  an  increase  in  the  mean  number 
of  publications  with  age  in  each  merit  index  group. 

The  mean  publication  score  of  scientists  by  age 
of  scientist  and  merit  index  is  given  in  table  43. 
Here  again,  with  few  exceptions  the  mean  pub- 
lication score  decreases  with  decreases  in  merit  in- 
dex and  increases  with  increases  in  age.  It  appears 
that  scientists  over  64  years  of  age  may  have  been 
more  conservative  in  scoring  their  publications 
than  younger  scientists. 

The  mean  number  of  publications  and  the  mean 
publication  score  for  all  ARS  scientists  by  merit 
index  are  given  in  table  44.  Those  scientists  in  the 
top  10  percentile  of  salary  for  their  age  produced 
six  times  as  many  publications  and  had  publica- 
tion scores  eight  times  those  of  scientists  in  the  bot- 
tom 10  percentile  of  salary  for  their  age. 

The  regularity  of  the  increase  in  productivity 
with  increase  in  salary  for  age  is  shown  in  figure 
5.  All  means  except  those  for  the  lowest  merit 
index  group  lie  in  a  straight  line  on  a  logarithmic 
scale.  The  scientists  in  the  lowest  10  percentile  of 
salary  for  their  age  are  not  producing  as  much  as 
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they  should  to  be  consistent  with  the  other  merit 
index  groups.  It  is  apparent  that  AES  pays  its 
scientists  in  accordance  with  their  productivity. 
Also,  for  scientists  of  the  same  age,  AES  gets  more 


for  its  money  from  those  currently  in  the  highest 
levels  of  salary,  because  the  spread  in  productivity 
is  much  wider  than  the  spread  in  salary. 


Table  41. — Correlation  coefficients  between  merit  index  June  SO,   1965  l  and  number  of  publications, 
publication  score,  and  highest  rank  order  on  a  single  publication  by  age 


Number  of  publications 

Publication  score 

Highest  rank  order 

Age,  June  30,  1965 

Number  of 
scientists 

r2 

Number  of 
scientists 

r2 

Number  of 
scientists 

r2 

24 _      -  - 

16 
20 
33 
49 
57 
63 

82 
61 
88 

106 

113 
88 

119 
95 

112 
97 

109 

100 

102 

124 

136 

99 

79 

82 

70 

74 

76 
83 
58 
52 
48 
57 
50 
54 
46 
61 

54 
42 
31 
28 
29 
24 
19 
22 
S 
6 

0.412 
*.  547 

**,  444 

'.  201 

.  193 

**.  481 

**.  358 
**.  418 
**.  493 
**.  394 
**.  617 
**.  575 
**.  530 
**.  491 
**'.  493 
**.  566 

**.  362 

**.  482 
**.  516 
**.  418 
**.  531 
**.  657 
**.  565 
**.  599 
**.  618 
**.  656 

**.  617 

**.  613 
**.  665 
**.  463 
**.  683 
**.  462 
**.  517 
**.  666 
**.  620 
**.  493 

**.  677 

**.  647 

**.  497 

**.  588 

*.  438 

*.  514 

**.  680 

.310 

.344 

.  561 

14 
14 
29 
46 
49 
58 

73 
50 
73 
90 

101 
77 

104 
88 
99 
84 

97 
93 
93 
113 
126 
93 
74 
77 
59 
70 

72 
75 
57 
47 
44 
54 
46 
52 
44 
60 

50 
42 
31 
27 
25 
22 
17 
22 
8 
4 

0.395 
*.  553 
*.  386 

**.  385 
.259 

**.  640 

**.  599 
**.  547 
**.  660 
**.  513 
**.  722 
**.  548 
**.  680 
**.  601 
**.  677 
**.  643 

**.  561 
**.  516 
**.  673 
**.  523 
**.  584 
**.  674 
**.  593 
**.  524 
**.  646 
**.  551 

**.  587 
**.  637 
**.  651 
**.  466 
**.  708 
**.  579 
**.  589 
**.  644 
**.  576 
**.  523 

**.  579 
**.  491 
*.  413 
**.  520 
**.  572 
**.  590 
**.  734 

.383 
-.202 

.  569 

2 
4 
11 
16 
35 
40 

52 
41 
62 
88 
85 
73 
102 
83 
98 
85 

98 
95 
91 
115 
123 
91 
74 
75 
64 
70 

71 
73 
53 
49 
44 
54 
46 
52 
43 
52 

52 
40 
30 
27 
27 
23 
15 
20 
8 
6 

0 

25   -        -    --  - 

.  630 

26  .  -  -   -      --.  . 

-.  182 

27 

**.  644 

28 -          --  .. 

.  193 

29       -  - 

**.  595 

30 

.216 

31 -   .   -   

.  142 

32 -  -       -  

.  243 

33 ---  -__ 

**.  318 

34    ... -_. 

**.  404 

35   ---  -_-  ~  --  -- 

**.  439 

36   -- .-_-___     .___. 

**.  318 

37   

**.  363 

38 -  

**.  369 

39 

**.  422 

40---  -- 

.  191 

41.  -  -  .-      

.  081 

42   ..-_--    ------- 

**.  455 

43 

.  137 

44   - 

.  091 

45._  -   -- -  -_ 

**.  458 

46.  -.  -  -- .  .-- 

.  162 

47 

**.  502 

48.  

**.  338 

49 

*.  266 

50--.  ..-  --    ---  ---   -  -. 

.225 

51 -.   .-  -- -  - 

**.  467 

52.-- -  -  ---  --- 

*.  319 

53 

.  191 

54_  

.250 

55_  - 

**.  429 

56.   -----  

*  350 

57.-.  ------  --. . 

*  288 

58. -  - 

.235 

59 .-- 

**.  403 

60-  ...  --  - 

**.  478 

61.  .-  

.  149 

62  ...  .  ...  

*  362 

63   

.221 

64 

.  175 

65.- 

*.  444 

66.  ...  -  

**.  693 

67 

.230 

68 

-.534 

69 

.396 

♦Significant  at  5-percent  level. 
♦♦Significant  at  1-percent  level. 


1  Sec  table  36,  footnote  1. 

2  Correlation  coefficient. 
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Table  42. — Mean  number  of  publications  to  Jan.  1,  1965,  by  age  of  scientists  and  by  merit  index  ' 


Mean  number  of  publications  by 

merit  index  group, 

June  30, 

L965 

Age  group  June  30,  1965 

0.9+  to 
1.0 

0.8+  to 
0.9 

0.7+  to 
0.8 

0.6+  to 
0.7 

0.5+  to 
0.6 

0.4+  to 
0.5 

0.3+  to 
0.4 

0.2+  to 
0.3 

0.1+  to 
0.2 

0  to  0.1 

24  to  28 

3.67 
11.71 
22.  24 
31.  35 
44.  72 
55.  56 
78.59 
78.72 
87.61 

3.47 
8.75 
16.  44 
22.  11 
31.  91 
49.  05 
54.  42 
62.80 
81.41 

4.86 
6.  45 
11.  69 
19.87 
32.73 
41.  94 
49.72 
58.  56 
53.27 

2.39 

5.70 

7.99 

17.  43 

28.57 

39.  60 

43.  67 
47.  16 

44.  01 

1.81 
4.  27 
7.  93 
16.  91 
20.  30 
30.  95 
36.  80 
54.77 
63.  04 

1.40 
5.23 
8.  00 
15.  33 
19.36 
22.  56 
43.  27 
34.76 
33.  04 

1.  16 
4.  25 
8.  73 
10.  95 
16.21 
17.  41 
33.46 
31.81 
38.53 

0.79 
4.31 
5.48 
10.  73 
14.  24 
20.  24 
24.  50 
23.  61 
53.36 

0.93 

2.  24 

5.  59 

13.  32 

10.  29 

15.  89 

16.  42 
20.42 
33.40 

0.33 

29  to  33 , 

1.42 

34  to  38.. 

2.28 

39  to  43.. 

5.30 

44  to  48. 

6.30 

49  to  53-. 

8.  12 

54  to  58 

9.  38 

59  to  63-.- 

19.47 

64  to  68.. 

16.  00 

1  See  table  36,  footnote  1. 


Table  43. — Mean  publication  score  to  Jan.  1,  1965,  by  age  of  scientists  and  by  merit  index  group,  Jurve 

30,  1965  l 


Age  group  June  30,  1965 


Mean  publication  score  by  merit  index  group,  June  30,  1965 


0.9+  to 
1.0 


0.8+  to 
0.9 


0.7+  to 
0.8 


0.6+  to 
0.7 


0.5+  to 
0.6 


0.4+  to 
0.5 


0.3+  to 
0.4 


0.2+  to 
0.3 


0.1+  to 
0.2 


0  to  0.1 


24  to  28 
29  to  33 
34  to  38 
39  to  43 
44  to  48 
49  to  53 
54  to  58 
59  to  63 
G4  to  68 


109 

188 

458 

606 

883 

971 

1,562 

1,313 

1,547 


48 

159 

290 

433 

559 

902 

1,357 

1,550 

1,  130 


58 
107 
228 
323 
663 
895 
911 
1,  112 
796 


23 
104 
139 
330 
534 
666 
756 
1,013 
776 


23 
72 
180 
309 
367 
508 
862 
1,065 
825 


23 
64 
156 
262 
365 
413 
534 
714 
770 


18 
53 
115 
204 
347 
352 
539 
589 
804 


11 

50 
91 
144 
232 
394 
447 
523 
542 


17 
26 
63 
205 
156 
287 
291 
405 
521 


5 

18 

35 

92 

85 

77 

119 

337 

315 


1  See  table  36,  footnote  1. 


Table  44. — Number  of  ARS  scientists,  mean  number  of  publications,  and  mean  publication  score  by  merit 

index  group,  June  80,  1965  l 


Merit  index  group,  June  30,  1965 

Item 

0.9+ 

to  1.0 

0.8  + 
to  0.9 

0.7  + 
to  0.8 

0.6  + 

to  0.7 

0.5  + 
to  0.6 

0.4  + 
to  0.5 

0.3  + 
to  0.4 

0.2  + 
to  0.3 

0.1  + 
to  0.2 

0  to 

0.1 

Total 

Number  of  scientists,  age  22  to  69. 

Mean  number  of  publications 

Mean  publication  score 

402 

39.  7 

757 

374 

29.  8 

607 

279 

28.  1 

541 

332 

23.  7 

446 

246 

20.  3 

392 

358 

17.  7 

314 

271 

14.  9 

276 

294 

13.3 

222 

292 

11.3 

183 

307 

6.  2 

94 

3,  155 

20.  5 

382 

1  See  table  36,  footnote  1. 
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Recalculation  of  the  Net  Change  in  Quality  of  ARS  Scientists  in  Two  Time  Periods 


In  tables  36  and  38,  the  net  change  in  quality 
of  ARS  scientists  was  calculated  using  arbitrary 
quality  factors  from  +  5  to  —  5  for  10  merit  index 
groups.  In  table  44  are  shown  actual  mean  pub- 
lication scores  for  each  merit  index  group.  It  was 
decided  to  recalculate  net  quality  changes  in  ARS 
scientists  using  actual  publication  scores  as  the 
measure  of  quality  factor  for  each  merit  index 
group.  The  quality  factor  was  calculated  as  fol- 
lows: 


Mean  publication  score 
of  merit  index  group 


ARS  mean 
publication  score 


100 

or  for  merit  index  group  0.9+  to  1.0: 
757-382 


100 


-=3.75 


The  results  of  the  recalculation  of  net  quality 
changes  using  real  quality  factors  are  shown  in 
table  45.  It  can  be  seen  that  the  same  conclusions 
are  reached  using  real  quality  factors  as  were 
reached  using  the  arbitrary  quality  factors  of  +  5 
to  —  5  for  the  various  merit  index  groups.  It  can 
now  be  stated  with  confidence  that  there  was  a 
small  net  decrease  in  quality  of  ARS  scientists  in 
the  period  June  30,  1956,  to  June  30,  1960,  and  a 
small  net  increase  in  quality  in  the  period  June  30, 
1960,  to  June  30,  1965. 


0       0.1+     0.2+     0.3+     0.4+    0.5+     0.6+    0.7+    0.8+  0.9+ 

to        to         to          to         to         to         to        to         to  to 

0.1      0.2        0.3        0.4       0.5       0.6       0.7       0.8       0.9  1.0 

MERIT  INDEX 

Figure  5. — Relationship  between  merit  index  and    (A) 

mean  number  of  publications  and   (B)   mean  publica- 
tion score. 


Table  45. — Net  change  in  number  of  scientists  in  2  periods  by  merit  index  group  l  and  net  change  in  quality 

oj  scientists  in  the  same  2  periods 


Net  change  in  number  of  scientists,  by  merit  index  group  and  associated 

quality  factors 2 

Net  change 
in  quality 

Period 

0.9  + 
to    1.  0 

(3.  75) 

0.8  + 
to   0.  9 

(2.  25) 

0.7  + 
to   0.  8 
(1.59) 

0.  6  + 
to   0.  7 
(0.  64) 

0.5  + 
to   0.  6 
(0.  10) 

0.4  + 

to  0.  5 

(-0.68) 

0.  3  + 

to  0.4 

(-1.06) 

0.2  + 
to  0.  3 
(-1.60) 

0.  1  + 

to  0.  2 

(-1.99) 

0  to 
0.  1 

(-2.88) 

of  ARS 
scien- 
tists 3 

June  30,  1956,  to 

June  30,  1960 

June  30,  1960,  to 

June  30,  1965 

57 
76 

52 
96 

54 
51 

68 
56 

72 
27 

33 

7S 

51 
59 

72 
61 

52 
60 

74 
70 

-40.  97 
86.45 

1  See  table  36,  footnote  1. 

2  Quality  factors  (figures  in  parentheses)  were  determined  from  the  mean  publication  scores  for  each  merit  index 
group  as  follows: 

Mean  publication  score  of  merit  index  group  —  ARS  mean  publication  score 

100 

'  Summation  of  the  products  of  number  of  scientists  times  the  quality  factor  for  the  10  merit  index  groups. 
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Correlation  Between  Merit  Index  and  Publications  for  Two  Time  Periods 


Correlation  coefficients  were  calculated  between 
merit  index  June  30,  1956,  and  (1)  publications 
per  year  to  January  1,  1955,  and  (2)  publication 
score  per  year  to  January  1,  1955.  Also,  correla- 
tion coefficients  were  calculated  between  merit  in- 
dex June  30,  1965,  and  (1)  publications  per  year 
to  January  1,  1965,  and  (2)  publication  score  per 
year  to  January  1,  1965.  The  results  are  shown 
in  table  46.  The  correlation  coefficients  were 
highly  significant  in  both  time  periods.  Merit 
index  was  more  highly  correlated  with  publica- 
tion score  per  year  than  with  publications  per 
year.  Both  publication  measures  were  more 
highly  correlated  with  merit  index  in  the  latter 
time  period  than  in  the  earlier  one. 


Table  46. — Correlation  coefficients  between  merit 
index  and  2  publication  measures  jor  2  periods  of 
time  for  the  1,327  scientists  employed  continuously 
by  ARS  since  Jan.  1,  1955 


Measures  compared 


Merit  index  June  30,  1956,  and  publications 
per  year  to  Jan.  1,  1955 

Merit  index  June  30,  1956,  and  publication 
score  per  year  to  Jan.  1,  1955 

Merit  index  June  30,  1965,  and  publications 
per  year  to  Jan.  1,  1965 

Merit  index  June  30,  1965,  and  publication 
score  per  year  to  Jan.  1,  1965 

**Significant  at  1-percent  level. 


Correlation 
coefficient 


**0.  339 
**.  389 
**.  497 
**.  542 


Peer  Evaluation  of  Research  Publications  Compared  With  a  Scientist's 
Own  Evaluation  of  His  Publications 


Publications  of  89  scientists  selected  at  random 
were  evaluated  by  peer  groups.  The  peer  groups 
were  composed  of  the  members  of  10  Division 
Personnel  Evaluation  Committees.  Each  commit- 
tee evaluated  publications  of  up  to  11  scientists 
in  its  own  division.  The  members  of  each  com- 
mittee are  senior  scientists  representing  the  disci- 
plines within  a  division.  Thus,  in  the  Crops  Re- 
search Division  one  would  likely  find  geneticists, 
physiologists,  pathologists,  horticulturists,  and 
agronomists  on  the  Division  Personnel  Evaluation 
Committee.  In  a  strict  sense  this  does  not  repre- 
sent a  peer  group  for  the  review  of  publications  in 
genetics  or  any  other  single  discipline  in  the  Crops 
Research  Division.  A  strict  definition  of  a  peer 
group  would  require  that  all  peers  evaluating 
genetics  publications  be  geneticists,  and  that  peers 
evaluating  pathology  publications  be  pathologists. 

The  situation  described  in  the  Crops  Research 
Division  is  the  most  diverse  of  all  the  divisions. 
In  three  other  divisions  where  peer  ratings  were 
made,  the  discipline  diversity  approaches  that  of 
Crops.  In  six  divisions  the  disciplines  are  more 
closely  related.  Nevertheless,  it  was  decided  that, 
since  these  committees  constitute  the  mechanism 
by  which  scientists  are  currently  evaluated,  the 


members  of  these  committees  could  be  called  peers 
for  the  purposes  of  this  study.  Since  each  member 
of  a  committee  was  asked  to  evaluate  independ- 
ently the  publications  of  the  selected  scientists,  it 
was  expected  that  the  ratings  given  a  scientist  by 
four  to  seven  raters  would  have  a  wider  variation 
than  would  likely  be  the  case  with  a  strict  peer 
group. 

Of  the  89  sets  of  publications  rated  by  peers, 
only  36  could  be  compared  with  the  scientist's  own 
evaluation.  The  reason  for  this  was  that  the  peers 
and  the  scientist  rated  a  different  number  of  pub- 
lications in  the  other  53  cases. 

Results  from  the  36  scientists  who  rated  the  same 
number  of  publications  as  were  rated  by  peers  are 
shown  in  table  47.  In  13  cases  the  scientist's  own 
publication  score  fell  within  the  range  of  peer 
scores.  In  four  of  these  the  scientist's  own  score 
was  lower  than  the  mean  peer  score.  In  an  addi- 
tional six  cases  the  scientist's  own  publication  score 
did  not  exceed  the  highest  peer  score  by  more  than 
10  percent.  Thus,  in  19  out  of  36  cases  the  scien- 
tist's own  evaluation  of  his  publications  and  peer 
evaluations  were  reasonably  related.  In  the  other 
17  cases  personal  evaluations  were  higher  than 
peer  evaluations.     However,  in  only  5  of  the  36 


40 


MISCELLANEOUS   PUBLICATION    1041,    U.S.    DEPARTMENT    OF   AGRICULTURE 


cases  did  the  ratio  of  personal  score  to  mean  peer 
score  exceed  2.  In  12  of  the  36  cases  the  ratio  of 
highest  peer  score  to  lowest  peer  score  exceeded  2. 
The  53  cases  rated  by  peers  that  were  not  com- 
parable with  personal  evaluations  are  shown  in 
table  48.  The  range  in  peer  scores  is  much  wider 
than  one  would  desire  in  many  cases.  However, 
in  the  column,  "Peer  mean  score  per  paper,"  it  is 
evident  that  differences  between  scientists  were 
shown.  In  order  to  see  if  there  was  a  relationship 
between  the  diversity  of  disciplines  represented  in 


a  division  and  the  range  in  individual  peer  ratings, 
table  49  was  prepared.  In  this  table,  the  scientists 
listed  in  both  tables  47  and  48  are  arranged  by  the 
peer  groups  that  evaluated  their  publications  so  as 
to  show  any  differences  between  peer  groups. 
(Differences  could  result  from  the  diversity  of 
disciplines  represented  by  the  scientists  in  a  divi- 
sion as  well  as  by  the  diversity  of  disciplines  rep- 
resented by  the  peer  group.)  The  last  column  in 
table  49,  giving  ratio  of  highest  score  to  lowest 
score,  shows  -the  diversity  in  peer  ratings. 


Table  47. — Peer  evaluation  of  research  publications  and  scientist's  evaluation  of  his  own  publications 


Scientist 

Number  of 
papers 

Peer  mean 
credit 

Personal 
credit 

Peer  mean 
score 

Personal 
score 

Range  in 

scores  among 

peers 

1 

49 

19 

11 

8 

2 

2 

12 

13 

1 

4 
14 

9 
24 

9 
25 
15 
16 

2 

4 

3 

3 

8 

8 

15 

36 

9 

11 

9 

4 

9 
6 
11 
30 
15 
21 
11 

20.8 
8.6 
2.6 
3.8 
1.9 
1.3 
7.6 
3.0 
.6 

2.  2 
5.9 
6.8 

16.9 
7.8 

11.7 
9.0 
9.  1 
1.8 
2.  0 

2.3 

2.  0 
4.  1 
3.3 
45 

10.  5 
4  2 
4.7 

3.  1 
1.7 

3.7 
4  7 
5.0 
11.9 
3.6 
7.7 
3.  0 

22.  0 
8.  75 
2.7 
3.8 
2.0 

1.  1 
7.6 
3.  1 
1.0 

3.2 

5.8 

7.3 

17.  1 

8.  1 

11.9 

10.  3 

8.9 

1.6 

2.0 

2.9 

2.  0 
5.4 
3.4 
5.  1 

10.  0 
4.4 
4  6 
4.8 
1.7 

3.6 
5.8 
6.8 
14  0 
3.9 
8.  0 
3.0 

618.6 
245.4 
51.5 
64  8 
33.3 
32.0 
163.8 
86.2 
30.4 

63.7 
172.0 
143.4 
298.3 
115.  1 
245.3 
215.8 
331.0 
15.  0 
42.3 

47.7 

68.0 
160.  5 

98.5 
162.5 
355.8 
145.4 
166.8 
165.0 

66.5 

206.3 
277.0 
210.  0 
513.  0 
138.0 
186.  5 
98.8 

1  496.  0 

2  308.  0 
113.0 
>  57.0 

50.0 
2  44  4 
236.  0 
1  94  0 
i  36.0 

«  35.0 
346.  0 

2  264  0 
900.0 
301.0 

1  310.  0 
394.0 

1  500.  0 

29.  0 
105.0 

95.  0 
'  87.0 
276.0 

2  139.  6 
615.0 

1  376.  5 
259.  5 
250.  0 

1  190.  0 
1  65.0 

1  222.  0 

1  355.  0 
360.  0 

2  615.  0 
219.  0 

2  236.  0 
142.5 

473.  0-779.  0 

2 

186.  0-294  0 

3 

21.  5-  81.  0 

4 

55.  5-  76.  0 

5 

6_            ..     .   -.   

30.  0-  40.  0 
19.  0-  42.  0 

7 

158.  5-164  0 

8 

60.  0-124  6 

9 .--         -   . -. 

15.  0-  42.  0 

10 

31.  0-110.  0 

11 

102.  0-282.  0 

12 

93.  0-258.  0 

13 

14 

15 

16 

196.  0-480.  0 

54  0-180.  0 

142.  0-363.  0 

145.  0-275.  0 

17 

18 

19 

20 

261.  0-500.  0 
15.  0-  15.  0 
33.  0-  50.  0 

34  0-  65.  0 

21.    

57.  0-  90.  0 

22 

23 

24 

25 

26 

27 

156.  0-165.  0 
74  0-136.  0 
108.  0-215.  0 
262.  0-430.  0 
103.  0-177.  5 
144  1-205.  0 

28 

95.  0-235.  0 

29 

30.. 

31 

32 

54  0-  79.  0 

169.  0-229.  0 
209.  0-360.  0 
159.  0-253.  0 

33 

34.. 

35 

455.  0-608.  0 
115.  0-157.  0 
107.  0-220.  0 

36 

92.0-112.0 

1  Personal  score  within  peer  range. 

2  Personal  score  outside  the  peer  range  but  does  not  exceed  the  highest  peer  score  by  more  than  10  percent. 
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Table  48. — Range  in  publication  scores  and  mean  publication  scores  oj  peers  for  scientists  whose  personal 

evaluation  could  not  be  compared  with  peer  evaluations 


Scientist 


L. 

2_. 
3. 
4_. 
5_. 
6.. 
7_. 
8_. 
9_. 

10. 
11 
12 
13 
14 
15 
16 
17. 
18. 
19 

20 
21 
22 
23 
24 
25 
26 
27 


Number 

Peer 

of 

mean 

papers 

score 

16 

255 

15 

163 

7 

104 

10 

114 

9 

124 

20 

270 

5 

87 

5 

127 

2 

71 

16 

323 

6 

102 

14 

248 

6 

188 

20 

196 

10 

195 

6 

77 

29 

277 

11 

145 

13 

195 

18 

400 

15 

144 

15 

305 

11 

131 

22 

268 

11 

52 

19 

107 

15 

116 

Range 
in  peer 
scores 


208-302 

151-174 

95-112 

105-122 

107-167 

247-321 

57-117 

94-168 

52-  96 

247-382 

48-154 

192-326 

98-270 

157-236 

160-214 

60-107 

202-331 

117-166 

175-236 

283-548 

116-166 

245-360 

101-163 

227-296 

39-  70 

65-175 

61-165 


Peer 
mean 
score 
per 
paper 


15.  9 
10.8 
14.  9 

11.  4 
13.8 
13.5 
17.4 
25.4 
35.5 

20.  2 
17.0 
17.  7 
31.  3 

9.8 
19.5 
12.8 

9.6 
13.  2 
15.0 

22.  2 

9.6 

20.3 

11.9 

12.  2 
4.7 
5.6 
7.7 


Scientist 


28 
29 

30 
31 
32 
33 
34 
35 
36 
37 
38 
39 

4(1 
41 
42 
13 
44 
45 
46 
47 
48 
49 

50 

51 
52 
53 


Number 

Peer 

of 

mean 

papers 

score 

1 

15 

1 

21 

18 

259 

3 

43 

10 

153 

15 

293 

15 

278 

9 

108 

9 

82 

11 

119 

8 

152 

13 

114 

6 

123 

16 

176 

9 

118 

18 

236 

8 

84 

15 

140 

13 

118 

3 

48 

11 

113 

13 

185 

11 

161 

7 

140 

16 

375 

4 

71 

Range 
in  peer 
scores 


12-  20 
10-  28 

186-366 
26-  82 
90-231 

183-448 

171-347 
57-166 
40-134 
69-183 

120-183 
61-192 

73-196 

125-288 

73-189 

136-408 

35-185 

63-249 

62-192 

17-  80 

56-260 

156-224 

149-180 

108-180 

325-428 

52-  93 


Peer 

mean 

score 

per 

paper 


15.  0 
21.  0 

14.  4 
14.3 

15.  5 
19.  5 
18.5 

12.  0 
9.  1 

10.8 

19.  0 
8.8 

20.  5 
11.  0 

13.  1 
13.  1 
10.  5 

9.3 
9.  1 

16.  0 
10.  3 
14.2 

14.6 
20.0 
23.4 
17.8 


Table  49. — Range  oj  peer  scores  on  the  publications  of  individual  scientists  by  peer  group 


Peer  group  ' 
and  scientist 


Group  1  (91,  96): 
1 


2.. 
3.- 

4.. 
5_- 
6.. 
7.. 
8.. 
9.. 
10. 


Number 

of 
papers 


49 
19 
29 
11 
13 
18 
15 
15 
11 
22 


Range  in 
scores 
among 
peers 


473-779 
186-294 
202-331 
117-166 
175-236 
283-548 
116-166 
245-360 
101-163 
227-296 


Ratio  of 
highest 

peer  score 
to  lowest 

peer  score 


1.  65 
1.58 
1.  64 
1.  42 
1.35 
1.  94 
1.43 
1.47 
1.  61 
1.  30 


Peer  group  1 
and  scientist 


Group  2  (77,  93) 

11 

12 

13 

14 

15 

16..- 

17 

18 ..-. 


Number 

of 
papers 


15 
16 
2 
4 
3 
3 
3 
11 


Range  in 
scores 
among 
peers 


145-275 

261-500 

15-  15 

33-  50 

34-  65 
57-  90 
17-  80 
56-260 


Ratio  of 

highest 

peer  score 

to  lowest 

peer  score 


1.  90 
1.  92 
1.  00 
1.  52 
1.  91 
1.58 
4.71 
4.  64 


See  footnote  at  end  of  table. 
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Table  49. — Range  of  peer  scores  on  the  publications  of  individual  scientists  by  peer  group  —Continued 


Peer  group  ' 
and  scientist 


Group  3  (78,  89) 

19 

20 

21 

22 

23 

24 

25 

26 

Group  4  (71,  95) 

27 

28 

29 

30 

31 

32 

33 

34 

Group  5  (76,  87) 

35 

36 

37 

38 

39 

40 

41 

42 

43 

Group  6  (68,  82) 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 


Number 

of 
papers 


11 
9 
4 

13 

11 
7 

16 
4 


11 

30 
15 

21 
II 
20 
111 
6 


9 

(i 
5 
5 

2 
Hi 

(I 
14 

6 


15 

36 

9 

16 

15 
7 

10 
9 

2() 


Range  in 
scores 
among 
peers 


144-205 

95-235 

54-  79 

156-224 

149-180 

108-180 

325-428 

52-  93 


159-253 
455-608 
115-157 
107-220 

92-112 
157-236 
160-214 

60-107 


169-229 

209-360 
57-117 
94-168 
52-  96 

247-382 
48-154 

192-326 
98-270 


108-215 
262-430 
103-178 
208-302 
151-174 

95-112 
105-122 
107-167 
247-321 
156-165 

74-136 


Ratio  of 

highest 

peer  score 

to  lowest 

peer  score 


1.  42 
2.47 
1.  46 
1.  44 
1.  21 
1.  67 
1.  32 
1.  79 


1.  59 
1.34 
1.  37 
2.06 
1.  22 
1.  50 
1.  34 
1.78 


1.  36 
1.73 

2.  05 
1.  79 
1.85 
1.55 

3.  21 

1.  70 

2.  76 


1.99 
1.  64 
1.  72 
1.  45 
1.  15 
1.  18 
1.  16 
1.56 
1.30 
I.  0(1 
1.84 


Peer  group  ' 
and  scientist 


Group  7  (45,  83)  : 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

Group  8  (45,  74): 

65 

66 

67 

68 

69 

70 

71 

Group  9  (47,  71) : 

72 

73 

74 

75 

76 

77 

78 

79 

Group  10  (28,  60) 

80 

81 

82 

83 

84 

85 

86 

87 

88 

89 


Number 

of 
papers 


14 

9 

24 

10 

15 

15 

9 

9 

11 


11 

8 

2 

2 

12 

11 

19 


13 
1 

4 
15 

1 

1 
18 

3 


9 
25 
13 

6 
16 

9 
18 

8 
15 
13 


Range  in 
scores 
among 
peers 


102-282 
93-258 

196-480 
90-231 

183-448 

171-347 
57-166 
40-134 
69-183 

120-183 


22-  81 
56-  76 
30-  40 
19-  42 
159-164 
39-  70 
65-175 


60-125 
15-  42 
31-110 
61-165 
12-  20 
10-  28 
186-366 
26-  82 


54-180 

142-363 

61-192 

73-196 

125-288 

73-189 

136-408 

35-185 

63-249 

62-192 


Ratio  of 

highest 

peer  score 

to  lowest 

peer  score 


2.76 
2.  77 
2.  45 
2.  57 
2.45 
2.  03 
2.  91 
3.35 
2.  65 
1.53 


3.77 
1.37 

1.  33 

2.  21 
1.  03 

1.  79 

2.  69 


2.  08 
2.  80 
3.55 

2.  70 
1.  67 
2.80 
1.97 

3.  15 


3.33 
2.56 
3.  15 
2.  68 
2.  30 

2.  59 

3.  00 
5.  29 
3.  95 
3.  10 


1  The  first  number  in  parentheses  shows  the  percentage  of  scientists  in  a  division  in  1  discipline;  the  second  figure  in 
parentheses  shows  the  percentage  of  scientists  in  3  disciplines. 


As  might  have  been  expected,  the  ratios  between 
highest  and  lowest  scores  of  peers  on  an  individual 
scientist's  publications  were  lower  in  those  divi- 
sions having  a  narrower  range  of  disciplines.  In 
the  first  6  divisions  shown  in  table  49,  the  publica- 
tions of  54  scientists  were  rated.  In  23  cases  the 
ratio  between  highest  and  lowest  peer  score  was 
1.50  or  under.  In  47  cases  the  ratio  was  under 
2.00,  and  in  only  7  cases  was  the  ratio  over  2.00. 
In  contrast,  in  the  last  four  divisions  shown  in 
table  49  where  disciplines  represented  in  a  division 


are  more  diverse,  the  ratings  of  individual  peers 
vary  widely.  In  these  last  4  divisions,  the  publica- 
tions of  35  scientists  were  rated.  In  only  three 
cases  were  the  ratios  between  the  highest  and  low- 
est peer  score  under  1.50,  and  in  only  seven  cases 
were  the  ratios  under  2.00.  In  28  cases  the  ratios 
were  over  2.00. 

Even  though  the  different  peers  in  a  group  vary 
widely  in  the  actual  publication  scores  they  give  to 
the  publications  of  one  scientist,  it  may  be  that  if 
10  scientists'  publications  are  evaluated,  the  differ- 
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ent  peers  would  place  scientists  in  the  same  rank 
order.  In  order  to  test  this  hypothesis,  it  is  neces- 
sary to  calculate  for  each  peer  the  publication  score 
per  year  given  each  scientist.  Then,  using  his 
publication  scores  per  year,  rank  the  scientists  from 
1  to  10.  This  ranking  can  be  compared  with  the 
rankings  given  by  other  peers  in  the  same  group 
for  the  same  10  scientists. 

In  order  to  compare  the  consistency  of  peer 
groups  in  ranking  scientists,  a  consistency  index 
has  been  formulated.  The  consistency  index  is 
calculated  as  follows : 

(a)  If  2  or  more  peers  give  a  scientist  the  same 
rank  order  among  10  scientists,  score  10 
points  each  for  the  highest  number  of 
peers  giving  the  same  rank  order.  For 
the  other  peers,  score  nine  points  if  their 
ranking  is  one  rank  order  away  from  the 
rank  order  given  by  the  highest  number 
of  peers,  eight  if  two  rank  orders  away, 
seven  if  three,  six  if  four,  five  if  five,  four 
if  six,  three  if  seven,  two  if  eight,  and  one 
if  nine. 

( b )  If  no  2  peers  give  a  scientist  the  same  rank 
order  among  10  scientists,  score  the  lowest 
ranking  as  10,  and  for  the  others,  score 
9  if  their  ranking  is  1  rank  order  away 
from  the  lowest  ranking,  8  if  2,  7  if  3,  6 
if  4,  5  if  5,  4  if  6,  3  if  7,  2  if  8,  and  1  if  9. 


Six  peer  groups  had  5  or  more  peers  evaluate 
the  publications  of  10  scientists.  The  evaluations 
of  the  first  five  peers  in  each  of  these  six  groups  are 
shown  in  tables  50  to  55.  The  tables  show  the 
publication  score  per  year  given  10  scientists  by 
each  of  5  peers  and  the  rank  order  given  these 
same  10  scientists  by  the  5  peers  based  on  the  pub- 
lication scores  per  year.  The  scientists  are  listed 
in  the  order  of  the  average  of  the  five  separate 
rankings.  The  peer  groups  in  tables  50  to  55  are 
designated  by  the  same  number  as  they  are  in  table 
49.  The  reason  that  a  different  number  of  scien- 
tists for  a  given  peer  group  appears  in  table  49  is 
that  in  table  49  only  those  scientists  who  were  rated 
on  the  same  number  of  publications  by  all  peers 
were  considered.  In  tables  50  to  55,  10  scientists 
were  considered.  Where  the  number  of  publica- 
tions differed  among  peers,  the  publication  scores 
were  adjusted  to  reflect  the  same  number  of  papers. 

It  is  evident  from  tables  50  to  55  that  when  5 
peers  follow  the  same  guidelines  to  independently 
evaluate  the  publications  of  10  scientists,  they  are 
remarkably  consistent  in  ranking  the  scientists. 
The  consistency  index  could  range  from  20,  where 
there  would  be  no  consistency  in  rankings,  to  50, 
where  there  would  be  perfect  agreement.  All  six 
peer  groups  had  a  consistency  index  above  45.  Al- 
though peer  groups  7,  8,  and  10  had  a  wider  range 
of  scores  among  peers  than  peer  groups  1,  2,  and  6, 
all  groups  were  equally  consistent  in  placing  sci- 
entists in  the  same  rank  order. 


Table  50. — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  1  and 

consistency  index 


Scientist 


Publication  score  per  year  given 
by  peer — 


D 


E 


Rank  order  given  by  pecr- 


D 


Rank  order 

consistency 

index 


9_. 
10. 


Average  for  group  1_ 


61.0 
28.3 
25.3 
24.5 
34.6 
23.  2 
25.  4 
18.3 
11.7 
10.  1 


56.5 
42.  5 
34  1 
31.  4 
21.9 
27.8 
29.  5 
17.5 
13.  6 
12.  5 


66.  6 
35.3 
41.  4 
36.  0 
31.  6 
29.  6 

22.  7 

23.  6 
15.  8 
16.3 


47. 
38. 
33. 
32. 
22. 
30. 
29. 
19. 
14. 
13. 


77.9 
54.8 
38.  3 
28.  4 
33.  4 
33.  2 
26.  5 
18.  7 
16.  6 
13.3 


1 
3 
5 

6 
2 

7 
4 
8 
9 
10 


1 
2 
3 
4 
7 
6 
5 
8 
9 
10 


1 
4 
2 
3 
5 
6 
8 
7 
10 
9 


1 
2 
3 
4 
7 
5 
6 
8 
9 
10 


1 
2 
3 
6 
4 
5 
7 
8 
9 
10 


1  Scientists  are  listed  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 


50 
47 
47 
45 
40 
47 
40 
49 
49 
49 


46.3 
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Table  51. — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  2  and 

consistency  index 


Scientist ' 

Publication  score  per  year  given 
by  peer — 

Rank  order  given  by  peer — 

Rank  order 
consistency 

A 

B 

C 

D 

E 

A 

B 

C 

D 

E 

index 

1 

2 

3 

3 

25.6 

14.  6 

15.  1 
15.  5 
13.3 

8.3 
11.8 
5.7 
2.7 
3.4 

36.3 

27.  5 

23.  0 

15.  8 

7.5 

12.  5 

7.0 

9.0 

8.0 

5.0 

13.8 
11.  2 

10.  7 
10.  5 
6.  5 
6.3 
5.  6 
3.7 
3.3 
4  0 

26.6 

22.  7 

17.  9 

26.  5 

14.  0 

11.  0 

8.5 

5.7 

3.5 

3.4 

65.  4 
46.  2 
43.8 
39.3 
17.  9 
23.3 
26.0 
10.  2 
7.5 
6.5 

1 

4 
3 
2 
5 
7 
6 
8 
10 
9 

1 
2 
3 

4 
8 
5 
9 
6 
7 
10 

1 

2 
3 
4 
5 
6 
7 
9 

10 
8 

1 
3 
4 
2 
5 
6 
7 
8 
9 
10 

1 
2 
3 

4 
7 
6 
5 
8 
9 
10 

50 
47 
49 
46 

5.     .-   -._           ..     - 

45 

5 

7 

8 

9 

10 

48 
45 
47 
46 

47 

Average  for  group  2 

47.  0 

1  Scientists  are  listed  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 


Table  52. — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  6  and 

consistency  index 


Scientist 1 

Publication  score  per  year  given 
by  peer — 

Rank  order  given 

by  peer — 

Rank  order 
consistency 

A 

B 

C 

D 

E 

A 

B 

C 

D 

E 

index 

1 .   ...   _ 

64.8 
38.6 
26.4 
28.  1 
20.8 
15.  4 
17.8 
20.4 
13.6 
11.2 

77.7 
34.  0 
25.6 
30.  2 
18.  4 
30.  1 
14.  0 
20.9 
10.  9 
9.5 

34.  8 
26.2 
16.3 
16.  1 
15.  1 
11.  8 
10.8 
14.  0 
7.4 
7.  4 

68.  0 
43.  0 
25.2 

20.  8 
12.9 

IS.  (1 

21.  5 
14.  0 

10.  8 

11.  0 

61.  0 
36.4 
32.  1 
23.6 
22.2 
17.  1 
19.8 
14.2 
14.8 
10.3 

1 
2 
4 
3 

5 
8 
7 
6 
9 
10 

1 

2 
5 
3 

7 
4 
8 
6 
9 
10 

1 
2 
3 
4 
5 
7 
8 
6 
9 
9 

1 
2 
3 
5 
8 
6 
4 
7 
10 
9 

1 
2 

3 
4 
5 
7 
6 
9 
8 
10 

50 

2  .    ..          ...        ..... 

50 

3 

47 

4 ...                  ... 

47 

5  ... .... 

45 

6 

45 

7 

43 

8.     ..---. 

46 

9  -.---.-   .   

48 

10 ...   

48 

Average  for  group  6 

46.9 

1  Scientists  are  listed  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 
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Table  53 — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  7  and 

consistency  index 


Scientist ' 

Publication  score  per  year  given 
by  peer — 

Rank  ordt 

r  given 

by  peer — 

Rank  order 
consistency 

A 

B 

C 

D 

E 

A 

B 

C 

D 

E 

index 

1        

24.7 

26.  0 

26.8 

48.0 

16.  6 

12.  2 

12.  0 

9.9 

8.0 

6.  1 

32.8 
34.6 
29.7 
25.8 
20.  0 
15.9 
15.8 
14.  6 
16.  6 
13.4 

21.  1 

17.  1 

17.  1 

19.  6 

10.  2 

9.0 

9.7 

6.9 

5.7 

5.0 

44.8 
39.  0 
32.  0 
36.0 
28.  2 
23.  1 
25.8 
18.3 
16.  3 
12.4 

28.  6 
25.0 
30.3 
20.  0 
19.  0 
16.6 
10.  9 
10.8 
9.9 
4.0 

4 
3 
2 
1 
5 
6 
7 
8 
9 
10 

2 
1 
3 
4 
5 
7 
8 
9 
6 
10 

1 
3 
3 

2 
5 
7 
6 
8 
9 
10 

1 
2 
4 
3 
5 
7 
6 
8 
9 
10 

2 
3 
1 
4 
5 
6 
7 
8 
9 
10 

46 

2 

47 

3 

46 

4                        

44 

5 

50 

6..        -.     

48 

7__          

47 

8.-. 

49 

9 

47 

10..      

50 

Average  for  group  7 

47.  4 

1  Scientists  are  lusted  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 


Table  54. — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  8  and 

consistency  index 


Scientist  ' 

Publication  score  per  year  given 
by  peer — 

Rank  order  given  by  peer — 

Rank  order 
consistency 

A 

B 

C 

D 

E 

A 

B 

C 

D 

E 

index 

1..   ..   ... 

28.8 

35.0 

23.6 

17.8 

9.  5 

15.2 

8.7 

9.7 

7.5 

7.  1 

41.  5 
45.  6 
22.  8 
20.3 
13.9 
15.  5 
11.6 

9.4 
10.0 

3.7 

34.6 

14.  0 

26.9 

19.  9 

16.  0 

12.  6 

5.7 

5.4 

7.5 

10.3 

28.0 
23.  7 
23.6 
19.  1 
16.9 

11.  1 
7.  7 

12.  0 
10.  0 

6.3 

9.0 
13.  0 
10.  6 
5.9 
7.7 
3.8 
3.  1 
2.  0 
2.5 
3.4 

2 
1 
3 
4 

7 
5 
8 
6 
9 
10 

2 
1 
3 
4 
6 
5 
7 
9 
8 
10 

1 
5 
2 
3 
4 
6 
9 
10 
8 
7 

1 

2 
3 
4 
5 
7 
9 
6 
8 
10 

3 
1 
2 
5 
4 
6 
8 
10 
9 
7 

47 

2...    

45 

3.   ...   - 

48 

4 

48 

5 

44 

6 

47 

7 

46 

7 

41 

9 . 

48 

10...   - 

44 

Average  for  group  8 

45.8 

1  Scientists  are  listed  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 
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Table  55. — Publication  score  per  year  and  rank  order  given  10  scientists  by  5  peers  in  peer  group  10  and 

consistency  index 


Scientist ' 

Publication  score  per  year  given 
by  peer — 

Rank  order  given  by  peer — 

Rank  order 
consistency 

A 

B 

C 

D 

E 

A 

B 

C 

D 

E 

index 

1 

2 . 

3 

4 

4 

6 

7 

8 

45.3 

24.3 

14.  2 

13.  1 

11.  7 

10.  5 

9.  3 

9.  1 

8.3 

8.4 

136.  0 
63.  0 
32.7 

24.  1 

25.  6 
18.  9 
17.  7 
12.  9 
14.8 
20.  6 

66.  3 
27.  3 
24.  2 
14.  0 
14.  1 
10.  5 
12.  6 
10.  5 
12.8 
5.7 

50.  3 

27.  3 

19.  7 

12.  5 

13.9 

7.6 

6.3 

7.3 

5.4 

5.  6 

70.7 
35.0 
20.  1 

13.  2 
8.9 

14.  0 

13.  3 

14.  5 
9.8 
3.8 

1 
2 
3 
4 
5 
6 
7 
8 
10 
9 

1 
2 
3 
5 
4 
7 
8 
10 
9 
6 

1 

2 
3 
5 

4 
8 
7 
8 
6 
10 

1 
2 
3 
5 
4 
6 
8 
7 

10 
9 

1 
2 
3 
7 
9 
5 
6 
4 
8 
10 

50 
50 
50 
47 
44 
46 
47 
43 

9 

10 

43 

44 

Average  for  group  10 

46.  4 

1  Scientists  are  listed  in  the  order  of  the  average  of  the  5  separate  rank  orders  given  by  the  peers. 


Some  of  the  Problems  Encountered  in  Evaluating  Research  Publications 


Most  scientists  agree  that  research  publications 
should  be  given  significant  weight  in  evaluations 
of  scientists  for  promotion  or  placement  actions. 
They  further  agree  that  quality  of  publications 
rather  than  quantity  of  publications  should  be  the 
decisive  factor  in  evaluating  scientists.  But  I 
found  no  agreement  among  scientists  on  how  qual- 
ity can  be  measured  or  who  should  do  the  measur- 
ing. While  scientists  generally  agree  that  the  dif- 
ferent authors  on  a  paper  with  multiple  authors 
deserve  different  amounts  of  credit,  they  see  great 
difficulties  with  any  system  that  apportions  the 
credit. 

Measuring  Quality 

One  of  the  principal  problems  in  measuring  the 
quality  of  a  publication  is  that  opinions  on  quality 
change  with  time.  Darwin's  "Origin  of  Species" 
is  given  a  much  higher  rating  today  than  it  was 
at  the  time  of  publication.  The  reverse  is  also  fre- 
quently true.  When  first  published,  a  given  paper 
may  appear  to  be  of  great  interest,  but  later  on  it  is 
shown  to  have  missed  the  mark.  For  this  reason, 
any  scheme  for  measuring  quality  must  provide 
for  reevaluations  periodically. 

A  second  problem  in  measuring  quality  is  that 
of  assigning  a  number  to  a  publication  as  a  meas- 


ure of  its  quality,  as  was  done  in  the  present  study. 
The  respondents  to  the  questionnaire  cooperated  in 
this  study  on  the  assurance  that  the  numbers  would 
not  be  used  in  evaluations  of  themselves.  Most 
scientists  believe  that  they  themselves  or  a  group 
of  peers  can  decide  whether  a  paper  is  outstand- 
ing, average,  or  minor,  but  they  feel  that  assigning 
one  paper  a  rank  order  of  30  and  another  a  rank 
order  of  35  puts  a  cloak  of  too  much  objectivity 
on  subjective  judgment. 

Because  of  the  difficulties  encountered  in  meas- 
uring quality,  it  was  decided  to  explore  how  peers 
would  rank  the  same  scientists,  first  on  the  basis 
of  publications  per  year  and  second  on  the  basis  of 
publication  score  per  year,  which  includes  quality 
along  with  numbers.  The  results  are  shown  in 
table  56.  There  is  reasonable  agreement  in  the 
rankings.  Out  of  the  60  scientists  ranked,  16  were 
given  the  same  ranking  by  both  methods,  8  were 
ranked  one-half  order  different,  14  were  ranked  1 
order  different,  8  were  ranked  iy2  orders  different, 
8  were  2  orders  different,  5  were  3  orders  different, 
and  1  was  4  orders  different. 

The  correlation  coefficient  between  publications 
per  year  and  publication  score  per  year  was  found 
to  be  0.71  for  all  scientists  in  the  study  in  grades  11 
through  15  on  the  basis  of  their  own  evaluations. 
It  will  also  be  recalled  from  earlier  sections  of  this 
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Table  56. — Ranking  of  10  scientists  by  6  peer  groups  by  n  umber  of  publications  per  year  and  by  publication 

score  per  year  given  by  peer  groups 


Scientist 

Rank  order  of  scientists  by 
peer  group  1  based  on — 

Rank  order  of  scientists  by 
poor  group  2  based  on — 

Rank  order  of  scientists  by 
peer  group  6  based  on — 

Publications 
per  year 

Publication 
score  per  year 

Publications 
per  year 

Publication 
score  per  year 

Publications 
per  year 

Publication 
score  per  year 

1 

2 

2 

1 

4 

9.5 

8 

6 

3 

7 

9.5 

5 

3 
1 

5 
9 

8 
2 
6 
4 
10 
7 

1 

2.5 

9 

4 

5.5 

2.5 

7 

5.5 

9 

9 

1 
2 
9 

3.5 
5.  5 
3.5 
5.5 
7 
10 
8 

8 
9 
4 
5 

1.  5 
10 
6.5 
6.5 
3 
1.  5 

6 
9 

3 

4 

4 

7 

5 

2 

6 

10 

7 

5 

8 

8 

9 

3 

10 

1 

Scientist 

Rank  order  of  scientists  by 
peer  group  7  based  on — 

Rank  order  of  scientists  by 
peer  group  8  based  on — 

Rank  order  of  scientists  by 
peer  group  10  based  on — 

Publications 
per  year 

Publication 
score  per  year 

Publications 
per  year 

Publication 
score  per  year 

Publications 
per  year 

Publication 
score  per  year 

1 

4 

9 

7 

2.5 

2.5 

9 

9 

1 

6 

5 

5 

7 
6 
1 
3 
9 
10 
4 
8 
2 

6 
5 
10 
3 
9 
7 
8 
2 
4 
1 

7.5 
6 
9 
5 

7.5 
4 
10 
2 
3 
1 

5 
10 
6 
2 
1 
8 
3 
9 
7 
4 

6 

2 

8 

3 

4.5 

4 

2 

5-. 

1 

6 

9 

7 

3 

8 

10 

9 

7 

10 

4.5 

report  that  the  correlations  of  publication  score 
per  year  with  promotions  and  merit  index  were 
only  slightly  better  than  the  correlations  between 
publications  per  year  and  promotions  and  merit 
index.  Thus,  it  would  appear  for  ARS  scientists, 
like  the  scientists  studied  by  Dennis  (1),  that  in 
general  the  scientists  who  produce  the  most  publi- 
cations are  likely  also  to  produce  more  papers  of 
high  quality.  There  are  exceptions  to  this,  of 
•course,  but  Dennis  found  that  among  19th  century 
scientists  who  produced  more  than  140  papers,  70 
percent  were  included  in  the  Encyclopaedia  Bri- 
tannica  as  eminent  scientists.  Fifty  percent  of 
those  who  produced  50  or  more  papers  were  in- 
cluded. Of  the  remaining  187  scientists  in  Dennis' 
study  who  produced  less  than  50  publications,  only 
6  were  included  in  the  Encyclopaedia  Britannica. 
Since  there  is  such  a  high  correlation  between 
quantity  and  quality,  it  would  seem  that  a  system 
which  could  divide  publications  into  five  classes  on 


the  basis  of  quality — outstanding,  very  important, 
important,  average,  and  acceptable — would  be  ade- 
quate for  purposes  of  evaluation. 

Dividing  Credit  Among  Multiple  Authors 

The  guide  used  in  the  questionnaire  to  divide 
credit  among  authors  brought  forth  several  objec- 
tions. Chief  among  these  was  that  it  would  dis- 
courage teamwoi'k  in  research.  This  is  a  valid 
objection.  If  an  author  knows  that  he  will  get  a 
credit  of  one  on  a  single-author  paper  and  only 
0.5  or  0.3  on  a  multiple-author  paper,  he  is  quite 
likely  to  favor  doing  research  by  himself.  An- 
other objection  to  the  guide  was  that  the  first- 
named  author  is  not  always  the  principal  contribu- 
tor to  the  research.  Many  scientists  try  to  encour- 
age associates  by  placing  the  associate  first  among 
the  authors  even  though  the  associate  was  not  the 
principal  contributor  to  the  research.    The  reverse 
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is  also  true  at  times.  Some  supervisors  take  first 
place  among  authors  even  though  a  subordinate 
was  the  principal  contributor.  Also,  some  scien- 
tists add  their  supervisor  to  the  author  list  as  a 
courtesy. 

Leaving  aside  these  objections,  dividing  credit 
among  authors  could  and  probably  would  lead  to 
serious  morale  problems.  Some  bitterness  arises 
now  in  deciding  the  order  in  which  authors  are 
listed.  If  a  higher  proportion  of  credit  were  as- 
signed to  those  first  listed,  this  bitterness  would  in- 
crease manyfold. 

Alternate  schemes  to  the  one  used  in  the  ques- 
tionnaire have  been  examined,  but  none  has  proved 
to  be  satisfactory. 

The  correlation  coefficient  between  publication 
credit  per  year  and  publication  score  per  year  was 
found  to  be  0.73  for  all  scientists  in  grades  11 
through  15  on  the  basis  of  their  own  evaluations. 
Since  this  was  almost  identical  to  the  correlation 
coefficient  of  0.71  between  publications  per  year 
and  publication  score  per  year,  it  is  concluded  that 
not  much  is  gained  by  dividing  credit  among 
multiple  authors.  Scientists  can  be  ranked  as  well 
by  numbers  of  publications  as  they  can  by  publica- 
tion credit. 

Who  Should  Measure  the  Quality  of  a  Scientist's 
Research  Publications  for  Promotion  Purposes? 

Most  scientists  agree  that  the  scientist  himself 
should  not  rate  his  own  papers,  but  many  feel  that 
he  should  take  part  in  the  measuring  process. 
Many  believe  that  the  immediate  supervisor  and 
others  close  to  the  work  should  do  the  rating. 
There  is  substantial  belief  that  a  paper  should  be 
rated  by  peers  in  the  discipline  represented  by  the 
paper.  All  these  suggestions  have  merit,  but  in  a 
large  organization  like  ARS  they  also  present 
many  problems. 

Consider  rating  by  peers  in  the  same  discipline. 
If  one  scientist's  papers  are  all  rated  by  the  same 
peers,  it  is  likely  that  his  papers  will  be  ranked 
in  proper  relationship  one  to  another.  But,  if  an- 
other scientist's  papers  are  rated  by  another  set  of 
peers,  it  is  quite  unlikely  that  the  two  scientists  can 
be  ranked  in  proper  relationship  to  each  other. 


To  get  a  proper  ranking  of  scientists,  all  papers  in 
a  given  field,  say  plant  genetics,  would  have  to  be 
rated  by  the  same  peers.  This  would  place  a  heavy 
burden  on  the  peers  assigned  to  do  the  rating. 
But,  even  if  such  a  scheme  could  be  worked  out, 
how  could  you  compare  scientists  in  plant  genetics 
witli  scientists  in  plant  physiology  when  the  plant 
physiologists'  publications  would  be  rated  by 
another  set  of  peers? 

Consider  rating  by  the  immediate  supervisor  and 
others  close  to  the  work.  The  supervisor  is  usually 
judged  on  the  productivity  of  his  unit.  It  would 
thus  be  to  his  advantage  to  give  high  ratings  to 
the  papers  of  his  scientists.  The  same  would  be 
true  for  ratings  by  other  scientists  in  the  same 
group.  Leaving  this  bias  aside,  it  is  quite  unlikely 
that,  with  the  large  number  of  supervisors  in  ARS, 
the  scientists  in  different  groups  would  be  placed 
in  proper  relationship  to  each  other. 

All  things  considered,  it  is  concluded  that  the 
peer  groups  currently  constituting  the  Division 
Personnel  Evaluation  Committees  are  in  the  best 
position  to  fairly  evaluate  the  quality  of  publica- 
tions of  all  scientists  in  a  division.  The  present 
peers  on  these  committees  are  all  senior  scientists 
familiar  with  their  division's  work  and  its  scien- 
tists. They  represent  the  various  disciplines  in  a 
division  and  can  thus  make  cross-discipline  com- 
parisons. They  will  not  be  biased  to  one  discipline 
or  one  research  group.  Instead,  they  are  in  a  posi- 
tion to  evaluate  a  given  publication  in  relation  to 
the  discipline  represented  by  the  publication  and 
also  in  relation  to  the  division's  whole  program. 

As  a  matter  of  fact,  the  members  of  these  com- 
mittees are  currently  judging  the  quality  of  re- 
search publications.  They  do  not  put  down  a  score 
for  each  publication  or  for  all  publications,  but  in 
arriving  at  a  consensus  of  the  merit  of  a  given 
scientist,  they  do  judge  the  quality  and  quantity 
of  his  publications  along  with  other  f  actors  as  they 
decide  that  scientist  A  ranks  above  scientist  B. 

If  more  formal  recognition  is  to  be  given  to  the 
quality  of  individual  publications,  which  I  think 
would  improve  the  evaluation  process,  the  system 
of  rating  publications  must  be  a  simple  one  since 
the  Division  Personnel  Evaluation  Committees  are 
already  overworked.  In  the  next  section  such  a 
plan  is  suggested. 
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A  Plan  for  Evaluating  Research  Publications 


The  plan  proposed  for  measuring  the  quality 
of  research  publications  adopts  the  quality  defini- 
tions for  degrees  A,  C,  and  P2  given  in  the  U.S. 
Civil  Service  Commission  Research  Grade-Evalu- 
at  ion  Guide,  Factor  IV:  Qualifications  and  scien- 
tific  contributions  (9).  Degrees  B  and  D  are  a 
part  of  the  plan  but  are  undefined  as  in  the  guide. 
The  plan  assumes  that  the  present  Division  Per- 
sonnel Evaluation  Committees  will  evaluate  the 
quality  of  a  scientist's  publications  in  the  normal 
course  of  their  deliberations  concerning  promo- 
tion and  placement. 

Scientific  publications  are  to  be  placed  in  one 
of  five  quality  classes  defined  in  reference  to  de- 
grees A,  B,  C,  D,  and  E,  as  follows. 

A.  Acceptable. — Fills  narrow  blanks  in  an 
existing  framework  of  knowledge,  or  corroborates 
existing  theory,  or  modifies  known  concepts  and 
techniques  to  deal  with  a  new  situation,  or  estab- 
lishes new  concepts,  techniques,  or  materials  of 
limited  interest,  or  restates  or  reviews  previously 
published  results  with  only  minor  additions  to  es- 
tablished knowledge. 

B.  Average. — Falls  between  classes  A  and  C. 

C.  Important. — Establishes  new  concepts,  tech- 
niques, or  materials  of  considerable  interest  and 
value  in  a  scientific  field  or  solves  a  problem  of 
considerable  value  to  science  or  the  public,  or  re- 
views, analyzes,  and  interprets  scientific  knowl- 
edge of  broad  scope. 


D.  Very  Important.- 

and  E. 


-Falls  between  classes  C 


E.  Outstanding. — Makes  a  major  advance  in  a 
scientific  field  and  opens  the  way  for  extensive 
further  developments  or  solves  a  problem  of  great 
importance  to  science  or  the  public. 

A  majority  of  all  publications  of  all  scientists, 
including  the  best  scientists,  will  be  expected  to 
fall  in  classes  A  and  B.  For  example,  three  of 
the  top  scientists  in  ARS  rated  their  own  papers 
as  follows: 


Class 

Scientist 

1 

2 

3 

E.  Outstanding 

0 

2 

16 

40 

13 

1 

1 

10 

36 

93 

1 

D.  Very  Important  _. 

3 

C.  Important . 

20 

B.  Average  _       _         

30 

A.  Acceptable - 

66 

Total  papers          

71 

141 

120 

It  is  believed  that  peers  would  have  given  the  pub- 
lications of  these  scientists  somewhat  higher  rat- 
ings than  the  scientists  gave  themselves. 

The  placement  of  a  scientist's  publications  in 
the  five  quality  classes  is  for  one  purpose  only, 
to  assist  the  committee  in  its  evaluation  of  the 
scientist.  No  records  of  placement  are  to  be  kept. 
Since  no  records  are  kept,  a  reevaluation  of  the 
scientist's  publications  will  be  made  the  next  time 
his  case  is  before  the  committee.  This  will  permit 
a  change  in  judgment  on  a  particular  paper  as  time 
passes. 

No  standards  can  be  set  on  how  many  papers 
in  a  given  class  would  qualify  a  scientist  to  be 
placed  in  a  given  degree  of  Part  IV  of  the  Research 
Grade-Evaluation  Guide  (9)  since  publications 
are  only  one  of  the  factors  considered  in  placement. 
But  ordinarily  one  would  expect  that  scientists 
who  are  rated  on  total  qualifications  and  achieve- 
ments (publications,  honors,  awards,  professional 
recognition,  etc.)  would  have  a  publication  pattern 
somewhat  as  follows : 

Those  given  degree  E  would  have  one  publica- 
tion 3  on  which  they  were  a  principal  contributor 
in  class  E  or  alternatively  two  or  more  publications 
in  class  D.  They  would  average  one  or  more  pub- 
lications per  year  and  have  at  least  10  publications 
in  the  combined  classes  C,  D,  and  E. 

Those  given  degree  D  would  have  one  publica- 
tion in  class  D  or  alternatively  three  or  more  pub- 


3  Wherever  the  word  "publication (s)"  is  used,  it  should 
be  interpreted  as  publications  or  patents,  or  both. 
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lications  in  class  C.  They  would  average  one  or 
more  publications  per  year  and  would  have  at  least 
10  publications  in  the  combined  classes  B,  C,  and  D. 
Those  given  degree  C  would  have  one  or  two 
publications  in  class  C  or  alternatively  four  or 
more  publications  in  class  B.     They  would  have 


at  least  six  publications  in  the  combined  classes 
A,  B,  and  C. 

Those  given  degree  B  would  have  one  to  three 
publications  in  class  B  or  alternatively  five  or 
more  publications  in  class  A. 

Those  given  degree  A  would  have  from  one  to 
four  publications  in  class  A. 
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Appendix 

UNITED  STATES  DEPARTMENT  OF  AGRICULTURE 
AGRICULTURAL  RESEARCH   SERVICE 

WASHINGTON,  D.C.    20250 
OFFICE  OF  ADMINISTRATOR  AllgUSt       16,        1965 

To:       Research  Scientists  and  Research  Supervisors  in  ARS 
From:     B.  T.  Shaw,  Assistant  to  Administrator 
Subject:   Evaluation  of  Scientific  Productivity 

First  of  all,  let  me  express  my  appreciation  of  the  contributions  you  have 
made  to  agricultural  research  and  of  your  efforts  to  make  ARS  an  outstanding 
organization.   I  also  want  to  thank  you  for  the  support  given  me  during  the 
years  I  was  Administrator.   I  am  sure  you  will  give  the  same  kind  of  support 
to  Drs.  Irving  and  Parker.   These  men  are  outstanding  scientists  and  they 
have  had  much  experience  in  research  administration.   They  will  work  to 
advance  agricultural  science  and  to  advance  your  interests  as  agricultural 
scientists. 

From  the  comments  I  have  had  from  many  of  you  and  from  my  own  observations, 
it  would  appear  that  we  might  be  able  to  work  out  more  objective  measures 
than  we  now  have  for  evaluating  research  productivity.   To  this  end,  I  am 
embarking  on  a  study  to  "Develop  more  reliable  criteria,  methods  and  tech- 
niques for  evaluating  qualifications,  achievements  and  professional  stature 
of  scientists  in  ARS."  As  I  see  it,  different  sets  of  values  are  involved 
in  the  following  classes  of  positions: 

(a)  a  research  scientist 

(b)  a  project  leader 

(c)  an  investigations  leader 

(d)  a  laboratory  director 

(e)  a  field  station  superintendent 

(f)  a  laboratory  chief 

(g)  a  branch  chief 

(h)   a  division  director 

Also,  different  sets  of  values  may  well  be  required  within  a  position  class 
to  evaluate  adequately  scientists  engaged  in  different  types  of  research. 
For  example,  it  seems  unlikely  that  the  same  rating  factors  would  be  equally 
valid  for  scientists  doing  beef  cattle  breeding  research,  scientists  doing 
plant  physiology  research  and  scientists  developing  wholesale  marketing 
facilities. 

For  a  scientist  working  alone,  rating  factors  should  be  based  on  his 
individual  contributions.   For  a  member  of  a  team,  rating  factors  should 
measure  his  contribution  to  and  stimulation  of  team  effort  as  well  as 
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individual  contributions.   In  rating  an  investigations  leader,  it  seems  to 
me  that  much  greater  weight  should  be  given  to  the  productivity  of  the 
investigations  group  than  to  his  individual  contributions.   Branch  chiefs 
and  division  directors  need  to  be  rated  on  economy  of  operations  and  other 
management  factors  as  well  as  on  the  research  output  of  their  unit  and 
their  individual  contributions.   For  all  scientists,  stature  as  a  "leader" 
in  a  field  of  science  or  segment  of  agriculture  or  industry  is  an  important 
rating  factor. 

Our  division  evaluation  committees,  composed  of  scientists  and  classification 
officers,  do  take  into  account  the  several  bases  for  judgment  that  I  have 
discussed  above.   I  have  been  well  pleased  with  the  progress  they  have  made 
in  refining  the  "man  in  the  job"  concept  of  position  classification.   It  is 
my  hope  that  in  the  study  I  am  undertaking  I  can  develop  some  objective 
measures  of  output  for  different  kinds  of  work  and  of  professional  standing 
that  will  assist  our  division  evaluation  committees  to  do  an  even  better 
job.   I  intend  to  make  a  number  of  studies  in  depth  on  various  rating 
factors,  starting  with  the  one  that  seems  to  be  most  troublesome. 

The  most  frequent  criticism  I  have  had  of  our  efforts  so  far  is  that  the 
evaluation  committees  rely  too  heavily  on  publications  and  more  particularly 
on  a  count  of  publications  in  making  evaluations  of  scientists.   I  want  to 
check  the  validity  of  this  criticism  and  also  ascertain,  if  possible,  the 
situations  in  which  publications  (with  appropriate  attention  to  quality)  can 
be  given  significant  weight  in  grade  determinations  and  the  situations  in 
which  they  cannot.   I  need  your  help  to  do  this,   I  will  need  to  divide  ARS 
scientists  into  homogeneous  groups  and  study  the  relation,  if  any,  between 
publications  (number  and  quality)  and  advancement.   I  shall  also  want  to 
consider  the  relation,  if  any,  between  publications  and  your  standing  in 
ARS  (as  measured  by  salary  and  age)  before  and  after  the  adoption  of  our 
current  evaluation  system.  A  questionnaire  is  attached  with  instructions 
for  filling  out  that  will  provide  the  necessary  information.   But  before 
you  turn  to  the  questionnaire,  I  want  to  discuss  some  of  the  guides  I  have 
set  up  for  answering  certain  questions. 

A.   RANKING  PUBLICATIONS 

Quantity  of  publications  presents  no  problems  for  measurement.   The  publi- 
cations can  be  counted.   Quality  is  an  entirely  different  matter.   It  is 
doubtful  that  a  single  quantitative  measure  can  be  found  that  will  adequately 
measure  all  facets  of  a  research  effort.   Perhaps  the  best  that  can  be  done 
is  to  list  a  group  of  papers  in  rank  order  from  the  most  to  the  least 
important.   An  individual  scientist  should  be  able  to  rank  his  own  papers 
in  this  way.   The  problem  becomes  more  difficult  when  the  publications  of 
several  scientists  are  to  be  placed  in  one  rank  order.   A  few  approaches 
to  the  solution  of  this  problem  have  been  suggested,  for  example: 

1.  Ranking  by  a  peer  group  of  scientists:   (a)  by  reading  all  the  papers; 
(b)  by  reading  abstracts  of  the  papers;  or  (c)  by  reading  titles. 

2.  Ranking  by   the  number  of  citations. 

3.  Ranking  by  the  place  of  publication. 
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Most  scientists,  I  believe,  would  accept  suggestion  1(a)  above  but  would 
have  reservations  on  the  others.   For  a  small  number  of  papers,  it  would  be 
feasible  to  use  a  peer  group  to  make  the  ranking  after  reading  all  papers. 
When  the  number  of  papers  becomes  large,  as  with  the  total  list  of  publica- 
tions of  the  3480  scientists  in  ARS,  this  procedure  has  to  be  discarded. 
So  also  does  suggestion  2  above.   Rather  than  to  use  the  method  in  suggestion 
3  above,  which  has  many  obvious  weaknesses,  I  believe  a  better  result  can  be 
obtained  by  having  each  scientist  place  his  papers  in  a  single  ARS  rank  order. 

I  recognize  that  ranking  one's  own  efforts  in  relation  to  the  efforts  of  all 
other  ARS  scientists  presents  opportunities  for  bias  in  both  directions. 
Some  of  our  people  will  be  too  modest  and  others  may  overrate  their  papers. 
But  after  allowing  for  this  bias,  I  believe  that  with  some  guide  lines  and 
an  effort  on  the  part  of  all  scientists  to  see  their  work  as  others  see  it, 
we  can  develop  a  better  ranking  of  papers  this  way  than  by  alternative  methods. 

I  propose  that  we  give  top  rank  to  original  research  followed  by  reviews  of 
research  for  scientists  and  put  reviews  of  research  for  laymen  on  the  bottom 
end  of  the  ranking.   Let  us  visualize  pigeonholes  numbered  from  1  to  100. 
We  will  reserve  pigeonhole  number  100  for  the  really  outstanding  contribu- 
tions to  science,  such  as  Darwin's  "Origin  of  Species,"  and  place  other 
papers  in  descending  order  down  to  pigeonhole  number  1.   So  that  we  will  all 
use  the  same  frame  of  reference,  I  suggest  the  following  guide  to  ranking: 

Rank  Orders  100  to  81:   Original  research  having  very  great  impact  on 
science,  agriculture  or  general  welfare. 

Papers  falling  in  this  range  would  be  discussed  in  all  histories  of  science 
pertaining  to  the  given  field.   Also,  the  research  would  be  cited  in  the 
writings  of  most,  if  not  all ,  other  scientists  doing  related  research. 

Note:   I  have  examined  the  titles  and  the  place  of  publication  of  the  3545 
research  publications  coming  out  of  ARS  in  calendar  year  1964.   With  this 
background  and  with  limited  knowledge  of  the  work,  I  estimate  only  two  of 
the  3545  ARS  1964  publications  will  have  very  great  impact  on  science, 
agriculture  or  general  welfare. 

Rank  Orders  80  to  61 :   Original  research  having  great  impact  on  science, 
agriculture  or  general  welfare. 

These  papers  are  frequently  cited  in  histories  of  science  and  in  the  writings 
of  other  scientists  doing  related  research.   Authors  of  these  papers  will  be 
invited  to  present  their  work  at  scientific  symposiums  and  in  scientific 
review  publications  and  will  often  be  cited  for  awards  for  outstanding  work. 

Note:   I  estimate  that  82  of  the  3545  ARS  1964  publications  will  have  great 
impact  on  science,  agriculture  or  general  welfare. 

Rank  Orders  60  to  1:   Original  research  having  moderate  to  1 imited  impact 
on  science,  agriculture  or  general  welfare. 

Papers  in  the  top  of  this  range  would  be  cited  frequently  in  the  writings 
of  other  scientists.   Papers  at  the  bottom  of  the  range  would  be  cited 
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infrequently.  Most  research  reports  of  original  work  will  fall  in  this 
range — from  good  to  run-of-the-mill. 

Note:   I  estimate  that  2464  of  the  3545  ARS  1964  research  publications  will 
have  moderate  to  limited  impact  on  science,  agriculture  or  general  welfare. 

Rank  Orders  50  to  1 :   Reviews  of  research  for  scientists. 

It  will  be  noted  that  I  have  an  almost  complete  overlap  of  rank  orders  for 
this  group  of  papers  with  those  next  above.   This  is  for  the  reason  that 
outstanding  reviews  of  science  for  scientists  will  have  much  more  impact  on 
science,  agriculture  or  general  welfare  than  many  run-of-the-mill  reports 
of  original  work. 

Many  papers  at  national  and  particularly  international  meetings  are  reviews 
of  research  rather  than  presentations  of  original  work.   While  some  mono- 
graphs present  original  work,  many  others  are  reviews. 

Note:   I  estimate  that  507  of  the  3545  ARS  1964  research  publications  are 
research  reviews  for  scientists. 

Rank  Orders  40  to  1:   Reviews  of  research  for  laymen. 

Again,  I  have  provided  for  an  overlap  of  rank  orders  because  certain  reviews 
for  laymen  have  great  impact  on  agriculture  and  general  welfare. 

These  are  reports  on  research  to  the  users  (other  than  scientists)  of  research 
information.   At  the  top  of  the  range  would  be  outstanding  Farmers5 Bulletins 
or  similar  publications,  in  the  middle  would  be  non-technical  reviews  in 
trade  papers  and  similar  journals,  and  at  the  bottom  would  be  popular  news 
articles. 

Note:   I  estimate  that  490  of  the  3545  ARS  1964  research  publications  are 
reviews  of  research  for  laymen. 

I  am  adding  an  appendix  as  a  part  of  this  memorandum  which  gives  examples  of 
papers  in  the  several  rank  orders.   All  division  directors  assisted  me  in 
preparing  this  appendix,  but  in  making  one  ARS  listing  it  has  been  necessary 
in  certain  cases  to  modify  the  rankings  suggested  by  the  divisions.   Some 
rankings  were  raised  and  others  lowered.   As  a  result,  I  take  full  responsi- 
bility for  the  rankings  shown.   Methods  papers  are  hard  to  rank  since  most 
methods  are  superseded  after  a  period  of  years.   The  rule  I  have  followed  is 
to  give  a  fairly  high  rank  to  methods  that  were  in  widespread  use  for  a 
period  of  years  after  publication.   Methods  not  generally  adopted  would 
rank  low. 

B.   PROPORTION  OF  CREDIT  TO  EACH  AUTHOR 

On  multiple  author  papers  one  should  divide  the  credit  in  a  wav  that  is 
fair  to  all.   The  easiest  way  is  to  divide  the  credit  equally.   In  my 
opinion,  for  most  papers  this  would  be  unfair  to  the  senior  author.   In 
those  situations  where  multiple  authors  take  turns  in  beinp  senior  author, 
dividinp  the  credit  equally  would  be  fair.   But  even  if  a  different 
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distribution  of  credit  among  authors  is  given,  those  who  take  turns  in 
being  senior  author  would  end  up  with  equal  scores  after  the  number  of 
papers  equalled  the  number  of  authors.   Granting  that  any  distribution  of 
credit  will  not  work  in  all  cases,  I  suggest  the  following  distribution  as 
a  guide.   However,  in  specific  cases  the  senior  author  may  deserve  0.9  of 
the  credit  and  in  other  cases  only  0.1.   Each  author  should  select  the 
proportion  of  credit  that  his  contribution  merits. 

one  author  -  1.0 

2  authors  -  1st  .6,  2nd  .4 

3  authors  -  1st  .5,  2nd  .3,  3rd  .2 

4  authors  -  1st  .45,  2nd  .25,  3rd  .15,  4th  .15 

5  authors  -  1st  .4,  2nd  .2,  3rd  .15,  4th  .15,  5th  .1 

6  authors  -  1st  .35,  2nd  .2,  3rd  .15,  4th  .1,  5th  .1,  6th  .1 

7  authors  -  1st  .3,  2nd  .2,  3rd  .1,  4th  .1,  5th  .1,  6th  .1,  7th  .1 

8  authors  -  1st  .25,  2nd  .15,  3rd  .1,  4th  .1,  5th  .1,  6th  .1,  7th  .1,  8th  .1 

9  authors  -  1st  .2,  2nd  .1,  3rd  .1,  4th  .1,  5th  .1,  6th  .1,7th. 1,8th  .1,9th  .1 
More  than  9  authors  -  divide  credit  equally 

C.   QUESTIONNAIRE  INSTRUCTIONS  (ARS  FORM  T-24) 

There  are  two  copies  of  the  questionnaire  attached.   Both  are  perforated  so 
that  you  can  tear  them  out.   Complete  and  return  one  copy  and  retain  the 
other  copy  for  your  records.   The  following  instructions  are  given  for  only 
those  questions  that  are  not  self-explanatory. 

1.  Questions  6  and  7,  Division  Code  and  Branch  or  Laboratory  Code.   Select 

the  appropriate  code  numbers  from  the  following  list.  (For  example,  those 
employees  in  the  Cotton  Insects  Research  Branch  of  the  Entomology  Research 
Division  would  write  33  in  question  6  and  33-20  in  question  7.) 

Entomology  Research  Division 

Office  of  Director 

Pioneering  Research  Laboratories 

Apiculture  Research  Branch 

Cotton  Insects  Research  Branch 

Fruit  and  Vegetable  Insects  Research  Branch 

Grain  and  Forage  Insects  Research  Branch 

Insects  Affecting  Man  &  Animal  Research  Branch 

Insect  Identification  &  Parasite  Introduction  Research  Branch 

Pesticide  Chemicals  Research  Branch 

Metabolism  and  Radiation  Research  Laboratory 

Soil  &  Water  Conservation  Research  Division 

Office   of   Director 

Pioneering  Research  Laboratories 

Northeast  Branch 

Corn  Belt  Branch 

Southern  Branch 

Northern  Plains  Branch 

Southern  Plains  Branch 

Northwest  Branch 

Southwest  Branch 


33 

33 

01 

33 

10 

33 

15 

33 

20 

33 

25 

33 

30 

33 

35 

33 

40 

33 

45 

33 

50 

41 

41 

01 

41 

10 

41 

15 

41 

20 

41 

25 

41 

30 

41 

35 

41 

40 

41 

45 

56  MISCELLANEOUS   PUBLICATION    1041,    U.S.    DEPARTMENT    OF    AGRICULTURE 

34  Crops  Research  Division 

34  01  Office  of  Director 

34  10  Pioneering  Research  Laboratories 

34  15  National  Arboretum 

34  16  Tropical  and  Subarctic  Agricultural  Research  Programs 

34  20  Cereal  Crops  Research  Branch 

34  25  Cotton  and  Cordage  Fibers  Research  Branch 

34  30  Forage  and  Range  Research  Branch 

34  35  Fruit  and  Nut  C*ops  Research  Branch 

34  40  Oilseed  &  Industrial  Crops  Research  Branch 

34  45  Tobacco  and  Sugar  C^ops  Research  Branch 

34  50  Vegetables  and  Ornamentals  Research  Branch 

34  55  Crops  Protection  Research  Branch 

34  60  New  Crops  Research  Branch 

42  Agricultural  Engineering  Research  Division 

42  01  Office  of  Director 

42  10  Pioneering  Research  Laboratories 

42  15  Harvesting  and  Farm  Processing  Research  Branch 

42  20  Crop  Production  Engineering  Research  Branch 

42  25  Livestock  Engineering  &  Farm  Structures  Research  Branch 

42  30  Farm  Electrification  Research  Branch 

44  Animal  Husbandry  Research  Division 

44  01  Office  of  Director 

44  10  Pioneering  Research  Laboratories 

44  14  Chemical  Metabolism  Laboratory 

44  15  Meat  Quality  Laboratory 

44  16  Radiation  Biology  Laboratory 

44  20  Dairy  Cattle  Research  Branch 

44  25  Beef  Cattle  Research  Branch 

44  30  Swine  Research  Branch 

44  35  Sheep  and  Fur  Animal  Research  Branch 

44  40  Poultry  Research  Branch 

45  Animal  Disease  and  Parasite  Research  Division 
45  01  Office  of  Director 

45  15  Plum  Island  Animal  Disease  Laboratory 

45  20  National  Animal  Disease  Laboratory 

45  25  Beltsville  Parasitological  Laboratory 

45  30  Regional  Laboratories  and  Field  Stations 

45  35  Foreign  Field  Locations 

51  Market  Quality  Research  Division 

51  01  Office  of  Director 

51  10  Pioneering  Research  Laboratories 

51  15  Instrumentation  Research  Laboratory 

51  20  Field  Crops  and  Animal  Products  Research  Branch 

51  25  Horticultural  Crops  Research  Branch 

51  30  Stored-Product  Insects  Research  Branch 

63  Clothing  and  Housing  Research  Division 

63  01  Office  of  Director 

63  10  Clothing  and  Textiles  Laboratory 

63  15  Housing  and  Equipment  Laboratory 
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Northern  Utilization  Research  and  Development  Division 
Office  of  Director 
Pioneering  Research  Laboratories 
Cereal  Properties  Laboratories 
Cereal  Products  Laboratory 
Fermentation  Laboratory- 
Industrial  Crops  Laboratory 
Oilseeds  Crops  Laboratory 
Engineering  and  Development  Laboratory 

Southern  Utilization  Research  and  Development  Division 

Office  of  Director 

Pioneering  Research  Laboratories 

Cotton  Chemical  Reactions  Laboratory 

Cotton  Finishes  Laboratory 

Cotton  Mechanical  Laboratory 

Cotton  Physical  Properties  Laboratory 

Food  Crops  Laboratory 

Naval  Stores  Laboratory 

Oilseed  Crops  Laboratory 

Engineering  and  Development  Laboratory 

Fruit  and  Vegetable  Products  Laboratory 

Eastern  Utilization  Research  and  Development  Division 

Office  of  Director 

Pioneering  Research  Laboratories 

Animal  Fat  Products  Laboratory 

Animal  Fats  Properties  Laboratory 

Hides  and  Leather  Laboratory 

Dairy  Products  Laboratory 

Milk  Properties  Laboratory 

Meat  Laboratory 

Plant  Products  Laboratory 

Engineering  and  Development  Laboratory 

Western  Utilization  Research  and  Development  Division 

Office  of  Director 

Pioneering  Research  Laboratories 

Cereals  Laboratory 

Wool  and  Mohair  Laboratory 

Fruit  Laboratory 

Sub-Tropical  Fruit  Laboratory 

Vegetable  Laboratory 

Field  C^ops  Laboratory 

Poultry  Laboratory 

Pharmacology  Laboratory 

Engineering  and  Development  Laboratory 

52  Transportation  and  Facilities  Research  Division 

52  01  Office  of  Director 

52  03  Handling  and  Facilities  Research  Branch 

52  04  Transportation  Research  Branch 

52  05  Wholesaling  and  Retailing  Research  Branch 


71 

71 

01 

71 

15 

71 

20 

71 

25 

71 

30 

71 

35 

71 

40 

71 

45 

72 

72 

01 

72 

15 

72 

20 

72 

25 

72 

30 

72 

35 

72 

40 

72 

45 

72 

50 

72 

55 

72 

60 

73 

73 

01 

73 

15 

73 

20 

73 

25 

73 

30 

73 

35 

73 

40 

73 

45 

73 

50 

73 

55 

74 

74 

01 

74 

15 

74 

20 

74 

25 

74 

30 

74 

35 

74 

40 

74 

45 

74 

50 

74 

55 

74 

60 

61 

61 

01 

61 

15 

61 

20 

61 

25 

61 

30 

62 

62 

01 

62 

05 

62 

10 

62 

15 

62 

20 
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Human  Nutrition  Research  Division 
Office  of  Director 
Food  Composition  Laboratory 
Food  Quality  and  Use  Laboratory 
Experimental  Nutrition  Laboratory 
Human  Metabolism  Laboratory 

Consumer  &  Food  Economics  Research  Division 

Office  of  Director 

Survey  Statistics  Staff 

Food  Consumption  Branch 

Diet  Appraisal  Branch 

Family  Economics  Branch 

Questions  24  through  33,  Publication  Summary.   For  this  study  a  publica- 
tion is  a  paper  presenting  your  own  work  (original  research,  review, 
analysis,  evaluation,  etc.,  that  is  reproduced  and  distributed  to  50  or 
more  people.   Do  not  count  abstracts.   Publications  of  the  same  material 
in  two  or  more  places  shall  be  counted  only  once.   However,  popular 
articles  are  usually  based  on  previously  presented  technical  papers  and 
should  be  counted.   Patents  not  described  in  other  publications  should 
be  treated  as  publications.   Some  publications  will  be  a  combination  of 
original  research  and  reviews  or  analyses  of  other  research.   In  such 
cases,  rank  the  paper  in  the  range  most  appropriate  to  its  subject  matter 
content.   If  you  were  the  author  of  a  Federal  or  State  publication  where 
authorship  is  not  shown,  count  this  paper  as  one  of  your  own.   If  you 
were  an  author  on  a  regional  publication  where  authors  are  listed  alpha- 
betically, take. credit  according  to  your  contribution. 

Before  you  answer  these  questions,  I  suggest  you  prepare  tables  as  follows 
for  the  periods  involved  in  Questions  24,  25  and  27  or  30  and  31,  depending 
upon  which  set  you  answer . 

Then  from  the  tables  you  have  prepared,  fill  in  the  spaces  a  through  f  of 
Questions  24  through  29  or  30  through  32  in  the  Publication  Summary  of 
the  questionnaire. 


Period  Being  Summarized 

Publication  Title 

Credit 

Rank  Ord 

er 

Score 

Title 

0.3 

X 

40 

= 

12 

(1) 

(2) 

. 

(3) 

: 

(4) 

(1)  List  your  publications  during  this  period. 

(2)  Refer  to  section  B  and  using  your  own  judgment,  enter  the  appropriate 
credit  for  each  paper  in  the  credit  column. 

(3)  Refer  to  section  A  and  taking  account  of  the  rank  orders  of  examples 
in  the  appendix,  and  giving  due  consideration  to  future  impact  of 
recent  papers,  give  each  of  your  papers  its  proper  rank  order. 

(4)  The  credit  times  the  rank  order  equals  the  score. 
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D.   PROMOTION  HISTORY 

Prior  to  the  reorganization  of  ARS  in  1954,  scientists  were  in  different 
Bureaus  having  somewhat  varying  personnel  policies.   Since  1954,  except  for 
the  two  divisions  (Market  Quality  and  Transportation  and  Facilities)  which 
joined  ARS  in  1964,  we  have  had  uniform  personnel  policies  for  all  scien- 
tists.  These  policies  have  changed  since  1954,  but  they  have  changed 
uniformly  for  all.   As  a  consequence,  the  period  from  January  1,  1955  to 
January  1,  1965  represents  the  best  period  to  compare  publications  (numbers 
and  quality)  with  promotions  to  see  what,  if  any,  has  been  the  relationship. 


This  study  is  designed  to  help  you  and  those  who  follow  you  in  ARS.   It  will 
be  as  good  as  the  efforts  you  put  into  it.   I  urge  your  full  cooperation. 
All  information  furnished  will  be  treated  in  confidence  and  the  published 
analysis  will  not  reveal  names.   The  information  is  requested  for  a  statis- 
tical study  and  will  not  be  used  to  evaluate  any  individual.   Please  send 
completed  questionnaire  to  B.  T.  Shaw,  Assistant  to  Administrator,  Agricul- 
tural Research  Service,  Room  512  Cotton  Annex,  U.  S.  Department  of  Agriculture, 
Washington,  D.  C.  20250.   I  would  appreciate  having  the  questionnaire 
returned  as  soon  as  possible,  but  in  any  event  not  later  than  October  1, 
1965. 

I  appreciate  your  help. 


9,  I,  -^S/Uun/- 
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ATTACHMENT 

Rank  Orders  100-81:   Original  Research 

100  -  Gibbs,  J.  Willard.   1878.   On  the  Equilibrium  of  Heterogenous  Sub- 
stances.  Conn.  Acad.  Trans.  3,  pp.  108-248,  343-524. 

95  -  Beadle,  G.  W.  and  Tatum,  E.  L.   1941.  Genetic  Control  of  Biochemical 
Reactions  in  Neurospora.   Proc.  Nat.  Acad.  Sci.  27:  499-506. 

Buckingham,  Edgar.   1907.   Studies  on  the  Movement  of  Soil  Moisture. 
USDA  Bur.  Soils  Bull.  38. 

Garner,  E.  W.  and  Allard,  H.  A.   1920.   Effect  of  the  Relative  Length 
of  Day  and  Night  and  Other  Factors  of  the  Environment  on  Growth  and 
Reproduction  in  Plants.   Jour.  Agr.  Res.  18:  553-606,  illus. 

Hammond,  John.   1927.   The  Physiology  of  Reproduction  in  the  Cow. 
Cambridge  University  Press. 

Hilgard,  E.  W.   1860.   Report  on  the  Geology  and  Agriculture  of  the 
State  of  Mississippi.   391  pp.,  map.   Jackson,  Miss. 

Kirchner,  J.  G. ,  Miller,  J.  M.  and  Keller,  G.  J.   1951.   Separation 
and  Identification  of  Some  Terpenes  by  a  New  Chromatographic  Technique, 
Anal.  Chem.  23,  420. 

Knipling,  E.  F.   1955.   Possibilities  of  Insect  Control  or  Eradication 
Through  the  Use  of  Sexually  Sterile  Males.   Jour.  Econ.  Ent .  48  (4): 
459-462. 

Moyer,  A.  J.  May  25,  1948.  Method  for  Production  of  Penicillin. 
U.  S.  Patent  2,442,141. 

Smith,  Theobald  and  Kilborne,  Frederick  Lucius.   Investigations  into 
the  Nature,  Causation,  and  Prevention  of  Texas  or  Southern  Cattle 
Fever.   Bull.  (1),  USDA,  BAI . 

90  -  Atwater,  W.  0.  and  Bryant,  A.  P.   1896.   The  Chemical  Composition  of 
American  Foods.   Expt.  Sta.  Bui.  28.  Washington. 

Goodhue,  L.  D.  and  Sullivan,  W.  N.   1942.   The  Preparation  of  Insecti- 
cidal  Aerosols  by  the  Use  of  Liquified  Gases.   Bur.  Ent.  Plant  Quar. 
Pub.  ET  190. 

Hendricks,  S.  B.  and  Fry,  W.  H.   1930.   The  Results  of  X-Ray  and 
Microscopical  Examinations  of  Soil  Colloids.   Soil  Sci.  29:  457-479. 

Jones,  H.  A.  and  Clarke,  A.  E.   1943.   Inheritance  of  Male  Sterility 
in  the  Onion  and  the  Production  of  Hybrid  Seed.   Araer.  Soc .  Hort . 
Sci.  Proc.  43:  189-194,  illus. 

Reeves,  Richard  E.   1950.   The  Shape  of  Pyranoside  Rings.   J.  Am. 
Chem.  Soc.  72,  1499. 
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90  -  Smith,  E.  F.   1920.   Bacterial  Diseases  of  Plants.   Philadelphia. 

Swern,  Daniel.   1947.   Electronic  Interpretation  of  the  Reaction  of 
Olefins  with  Organic  Peracids.   Jour.  Am.  Chem.  Soc .  69,  pp.  1692-98. 

Wright,  S.   1921.   Systems  of  Mating.   Genetics  6:111. 

85   -  Atwater,  W.  0.   1895.   Methods  and  Results  of  Investigations  on  the 

Chemistry  and  Economy  of  Food.   Bui.  21,  Office  of  Experiment  Stations, 
USDA. 

Brody,  S.   1945.   Bioenergetics  and  Growth.   Reinhold  Publishing 
Company,  New  York. 

Brooks,  F.  A.,  et  al .   1952.   Heat  Transfers  in  Citrus  Orchards  Using 
Wind  Machines  for  Frost  Protection.   Agr.  Eng.  33,  Nos.  2  and  3,  pp. 
74-78,  143-147. 

Buck,  J.  M.   1930.   Studies  of  Vaccination  During  Calf hood  to  Prevent 
Bovine  Infectious  Abortion.   Jour.  Agr.  Res.  41,  p.  667. 

Corothers,  W.  A.  and  Hill,  J.  W.   1933.   Polymerization  and  Ring  Formu- 
lation XXII.  Stereo-Chemistry  and  Mechanism  in  the  Formation  and 
Stability  of  Large  Rings.   Jour.  Am.  Chem.  Soc.  55:  50  43-52. 

Gahan,  J.  B. ,  Travis,  B.  V.  and  Lindquist,  A.  W.   1945.   DDT  as  a 
Residual  Type  Treatment  to  Control  Anopheles  quadrimachlatus :  Practical 
Tests.   Jour.  Econ.  Ent .  38(2):  231. 

Kneeland,  Hildegarde,  and  staff  of  National  Resources  Comm.   1939. 
Consumer  Expenditures  in  the  United  States.  Washington. 
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16.  What  was  your  grade  on   1-1-65  (For  those  in  PL  313  or  other  non-GS  grades  use  the  GS  grade  I 
equivalent  to  your  salary)  GS- 

17.  How  many  years  have  you  been  publishing  research   (Account  for  your  full  career  in  research  wilhuut  regard  to  place  of  employ- 
menl    Begin  publication  record  with  the  dale  your  first  publication  was  printed  but  subtract  years  of  service  such  as  military. 


1 

a. 

M  or  more  years 

b. 

21  to  30  years 

c. 

11  to  20  years 

d. 

1  to  10  yeats 
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18.  ARS  employment  (Check  applicable  category  and  complete  the  questions  specified) 


a.  Employed  by  ARS  continuously  since   1-1-55   (Answer  only  questions   19  thru  29,   and  33) 


b.  Entered  ARS  after  1-1-55  -  including  those  in  Market  Quality  Division  and  Transportation  and  Facilities  Division 
(Answer  only  questions  30  thru  33) 


19.  What  was  your  grade  on   1-1-55  (For  those  in  PL  313  or  other  non-GS 
grades,  use  the  GS  grade  equivalent  to  your  salary) 


GS- 


20.     ANNUAL    SALARY    ON    6-30-56 


2-1.  Number  of  grade  changes  from  1-1-55  to  1-1-65  (From  grades  5  to  7,   7  to  9.  or  9  to  11  is  a  one-grade  change;   Grades 
9  to  14  are  4  changes;  Grades  13  to  15  are  2  changes) 

22.  Using  current  definitions  of  positions,  check  the  one  below  that  best  describes  your  position  on  1-1-55 


a.   Research  scientist  (All  scientists  in  Pioneering  Research  Laboratories  and  all  others  who  work  full-time  on  personal 
research  or  in  research  teams) 


b.  Project  leader 


c.  Investigations  leader 


d.  Laboratory  director  (Does  not  report  direct  to  Division  Director) 


e.  Field  station  superintendent 


f.  Laboratory  chief  (Reports  direct  to  Division  Director) 


g.  Branch  chief  (All  scientists  in  the  office  of  branch  chief,  i.e.  Assist  Chief,  Assist,  to  Chief,  etc.) 


h.  Division  director  (All  scientists  in  office  of  division  director,  i.e.  Assist.  Director,  Assist,  to  Director,  etc.) 


23. 

VI  r 

at  was  your  highest  earned  degree  on   1-1-55 

a.  PhD,  ScD  or  equivalent  degree 

b.  MS,  MA,  DVM  or  equivalent  degree 

c.  BS,  BA  or  equivalent  degree 

d.  No  degree 

PUBLICATION  SUMMARY 
(Read  Section  C,  instruction  2  before  completing  questions  24  thru  33.    After  preparing  the  tables  suggested  in  instruction  2,  use 
those  tables  to  complete  questions  24  thru  33) 


ITEMS 

EMPLOYED   BY   ARS  CONTINUOUSLY  SINCE    1-1-5S 
GIVE   YOUR    PUBLICATION   SUMMARY 

ENTERED   ARS   AFTER 

1-1-55  GIVE  YOUR 
PUBLICATION   SUMMARY 

(In  counting  No.  years  in  items  b,  d  &  I 
below,  subtract  years  in  military, 
school  or  Industry  where  you  were 
unabte  to  publish) 

PRIOR 

TO    ENTRY 

INTO 

ARS 

BETWEEN 
EN  TRY 

1-1-55 

TO 
1-1-55 

BETWEEN 
1-1-55 
AND 
1-1-65 

TO 

1-1-65 

BETWEEN 

EN  TRY 
INTO    ARS 
&     1-1-65 

PRIOR 

TO    ENTRY 

INTO 

ARS 

BETWEEN 

ENTRY 

AND 

1-1-65 

TO 

1-1-65 

o.  No.  publications    (Total  No.  papers 
in  the  period  on  which  your  name 
appears  as  author) 

24 

25 

26 

: 

27 

: 

28 

29 

(25+  27) 

r                   = 

b.  Publications  per  year  (Divide  No. 
publications  in  item  a  by  the  No. 
years  in  the  period) 

(26a  T  No. 

yrs.  in 

24  and  25) 

■  .'fin  "    No. 

yrs.   in 
26  and  27) 

(29a  ~  No. 

yrs.   in 
25  and  27) 

(32a  ~  No. 

yrs.  in 
30  and  31) 

c.  Publication  credit  for  period  (Sum 
of  credits  in  credit  column  of 
table  prepared) 

H 

: 

(25+  27) 

- 

: 

d.  Publication  credit  per  year  (Sum 
of  credits  in  item  c  divided  by  No. 
years  in  period) 

(26c-  No. 

yrs.  in 
24  and  25) 

(28  c  +  No. 

yrs.  in 
26  and  27) 

(29c  t  No. 
yrs.  in 
25  and  27) 

(32c  +  No. 

yrs.  in 
30  and  31) 

o.  Publication  score  for  period  (Sum 
of  scores  in  score  column  of  table 
prepared) 

h                 : 

-                       - 

(25+  27) 

- 

: 

(.  Publication  score  per  year  (Sum  of 
scores  in  item  e  divided  by  No. 
years  in  period) 

(26e  T  No. 

yrs.  in 

|  24  and  25) 

(2Se  T  No. 

yrs.  in 
26  and  27) 

(29e  ~  No. 

yrs.   in 
25  and  27) 

(32e  ~  No. 

yrs.   in 
30  and  31) 

33.  What  is  the  highest  rank  order  given  to  any  one  publication  on  which  you  were  sole 
or  senior  author 
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PUBLICATION  AND  PROMOTION  QUESTIONNAIRE 


1.   NAME  (Last,   tirat,  middle) 


2.    BIRTH    DATE   -   MONTH.    DAY.    YEAR   (Uae   alx  digits,    I.e.    04-30-29) 


3.    SEX 

MALE      □ 
FEMALE        □ 


YEAR    OF    ENTRY    INTO 
ARS   OR    ITS   PREDE- 
CESSOR   AGENCIES 


SELECT    FROM   CODE    LIST   (Section  C,   Inat.   1) 


6.    DIVISION    CODE 


7.    DIVISION    AND    BRANCH    OR    LABORATORY 
CODE 


8.   DUTY   STATION  CODE  (From  "Contact  Point"  on 
Time  and  Attendance  Report) 


STATE     TOWN 


9.   ANNUAL  SALARY   ON   6-30-60  (Only  II  employed  by  ARS  on  6-30-60) 

s 


10.  Did  you  have  full-time  employment  in  research  or  research  and  teaching  before  entering  ARS  or  its  predecessor  agencies? 

~" (Those  employees  in  the  two  marketing  research  divisions  who  transferred  to  ARS  on  July  1.  1964  will  answer 
Yes  "Yes"  even  though  some  were  in  a  predecessor  agency  to  ARS) 


No 


11.  If  "Yes"  in  question   10,  your  previous  employer  was:    (Check  appropriate  one;  if  your  previous  employment  was  in  two  or  more, 
check  "d.  Other") 


a. 

College,  University  or  State  experiment  station 

b 

industry 

c. 

Federal  Government 

d. 

Other 

12.  Ownership  or  control  of  facilities  in  which  you  work 


°.   ARS 


b.  Cooperator 


13. 

A 

e  the  facilities  in  which  you  work 

o.  On  or  near  (10  miles)  a  college  or  university  campus 

b.  More  than  10  miles  from  a  college  or  university  campus 

14.  What  was  you  highest  earned  degree  on   1-1-65 


o.  PhD,  ScD,  or  equivalent  degree 


b.  MS,  MA,  DVM,  or  equivalent  degree 


c.  BS,  BA,  or  equivalent  degree 


d.  No  degree 


15.  What  was  your  position  title  on   1-1-65 


0. 

Research  Scientist  (All  Scientists  in  Pioneering  Research  Laboratories  and  alt  others  who  work  full-time  on  personal 
research  or  in  research  teams) 

b. 

Project  leader 

c. 

Investigations  leader 

d. 

Laboratory  director  (Does  not  report  direct  to  Division  Director) 

e. 

Field  station  superintendent 

f. 

Laboratory  Chief  (Reports  direct  to  Division  Director) 

9- 

Branch  Chief    (All   scientists  in  the  office  of  branch  chief,  i.e.  Assist.  Chief,  Assist,  to  Chief,  etc.) 

h. 

Division  Director  (All  scientists  in  the  office  of  division  director,  i.e.  Assist.  Director,   Assist,  to  Director,  etc.) 

16.  What  was  your  grade  on   1-1-65  (For  those  in  PL  313  or  other  non-GS  grades  use  the  GS  grade  I 
equivalent  to  your  salary)  GS- 

17.  How  many  years  have  you  been  publishing  research   (Account  for  your  full  career  in  research  without  regard  to  place  of  employ- 
ment    Begin  publication  record  with  the  date  your  first  publication  was  printed  but  subtract  years  of  service  such  as  military, 
school  or  industry  where  \ou  were  unable  to  publish) __^_ 


o.  M  or  more  years 


b.   21  to  30  years 


c.   1  1   to  20  years 


d.   1  to  10  years 
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18.  ARS  employment  (Check  applicable  category  and  complete  the  questions  specified) 


a.  Employed  by  ARS  continuously  since  1-1-55   (Answer  only  questions   19  thru  29,  and  33) 


b.  Entered  ARS  after  1-1-55  -  including  those  in  Market  Quality  Division  and  Transportation  and  Facilities  Division 
(Answer  only  questions  30  thru  33) 


19.  What  was  your  grade  on  1-1-55  (For  those  in  PL  313  or  other  non-GS 
grades,  use  the  GS  grade  equivalent  to  your  salary) 


GS- 


20.     ANNUAL    SALARY    ON-6-30-56 


21.  Number  of  grade  changes  from  1-1-55  to   1-1-65  (From  grades  5  to  7.  7  to  9,  or  9  to  11  is  a  one-grade  change;   Grades 
9  to  14  are  4  changes;  Grades  13  to  15  are  2  changes) 

22.  Using  current  definitions  of  positions,  check  the  one  below  that  best  describes  your  position  on  1-1-55 


a.   Research  scientist  (All  scientists  in  Pioneering  Research  Laboratories  and  all  others  who  work  full-time  on  personal 
research  or  in  research  teams) 


b.  Project  leader 


c.  Investigations  leader 


d.  Laboratory  director  (Does  not  report  direct  to  Division  Director) 


e.  Field  station  superintendent 


f.  Laboratory  chief  (Reports  direct  to  Division  Director) 


g.  Branch  chief  (All  scientists  in  the  office  of  branch  chief,  i.e.  Assist  Chief,  Assist,  to  Chief,  etc.) 


h.  Division  director  (All  scientists  in  office  of  division  director,  i.e.  Assist.  Director,  Assist,  to  Director,  etc.) 


23. 

tth 

at  was  your  highest  earned  degree  on 

-1-55 

a.  PhD,  ScD  or  equivalent  degree 

b.  MS,  MA,  DVM  or  equivalent  degree 

c.  BS,  BA  or  equivalent  degree 

d.  No  degree 

PUBLICATION  SUMMARY 
(Read  Section  C.  instruction  2  before  completing  questions  24  thru  33.    After  preparing  the  tables  suggested  in  instruction  2,  use 


those  tables  to  complete  questions  24  thru  33) 

ITEMS 

EMPLOYED   BY    ARS  CONTINUOUSLY   SINCE    1-1-55 
GIVE  YOUR   PUBLICATION  SUMMARY 

ENTERED    ARS   AFTER 

1-1-SS  GIVE  YOUR 
PUBLICATION   SUMMARY 

(In  counting  No,  years  in  items  b,  d  St  i 
below,   subtract  years  in  military, 
school  or  Industry  where  you  were 
unable  to  publish) 

PRIOR 

TO    ENTRY 

INTO 

ARS 

BETWEEN 
ENTRY 

AND 
1-1-55 

TO 

1-1-55 

BETWEEN 
1-1-SS 
AND 
1-1-65 

TO 

1-1-65 

BETWEEN 

EN  TRY 
INTO    ARS 

&   1-1-65 

PRIOR 

TO    ENTRY 

IN  TO 

ARS 

BETWEEN 

ENTRY 

AND 

1-1-65 

TO 

1-1-65 

o.  No.  publications   (Total  No.  papers 
in  the  period  on  which  your  name 
appears  as  author) 

■\ 

25 

26 

27 

: 

28 

29 

(25+  27) 

r                   = 

b.  Publications  per  year  (Divide  No. 
publications  in  item  a  by  the  No. 
years  in  the  period) 

(26a  T  No. 

yrs.  In 

24  and  25) 

(28a  -   iVo. 

yrs.  In 
26  and  27) 

(29a  -r  No. 

yrs.   In 
25  and  2  7) 

(32a  ~  No. 

yrs.  In 
30  and  31) 

c.  Publication  credit  for  period  (Sum 
of  credits  in  credit  column  of 
table  prepared) 

4 

: 

(25+  27) 

■ 

- 

6.  Publication  credit  per  year  (Sum 
of  credits  in  item  c  divided  by  No. 
years  in  period) 

(26  c  +  No. 

yrs.   in 
24  and  25) 

(28  c  +  No. 

yrs.   In 
26  and  27) 

(29c  +  No. 
yrs.   in 
25  and  27) 

(32c  +  No. 

yrs.  in 
30  and  31) 

e.  Publication  score  for  period  (Sum 
of  scores  in  score  column  of  table 
prepared) 

- 

1-                  = 

- 

(25+  2  7) 

" 

- 

f.  Publication  score  per  year  (Sum  of 
scores  in  item  e  divided  by  No. 
years  in  period) 

(26e  ~  No. 

yrs.   In 
24  and  25) 

(28e  ~  No. 

yrs.   in 
26  and  27) 

(29e+  No. 

yrs.   in 
25  and  27) 

(32e  ~  No. 

yrs.  In 
30  and  31) 

33.  What  is  the  highest  rank  order  giv 

en  to  any  c 

ne  public: 

ition  on  wh 

ich  you  we 

re  sole 

or   senior  author 
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